This dissertation explores the development and implementation of Parallel Surrogate-based Optimization Algorithms which are designed specifically for problems with multiple local optima and require a large computation burden. With the help of multiple variants of Surrogate-based Optimization algorithms, different computationally expensive groundwater problems can be solved within a limited computation budget. In the first part of the dissertation, we examine parallelism performance of Parallel Stochastic Radial Basis Function (p-SRBF) Algorithm on groundwater management problems. SRBF uses Radial basis functions as a surrogate surface to assist the search for an optimal solution. In its revised Parallel version, p-SRBF is able to reach super-linear speed-up and reduce the computation budget by a large factor in the two problems we worked on. Both of the problems deal with management of pump and treat systems for remediation on contaminated Superfund sites. The analysis also shows that p-SRBF performs the best among all three popular parallel optimization algorithms compared, including Parallel Genetic Algorithm, Parallel NOMAD and APPSPACK. The second part concentrates in the area of constraint dealing strategy. The problem is based on a real world model simulating land subsidence induced by overdraft of groundwater. In order to help decision makers plan groundwater exploitation in an efficient way given the land subsidence and groundwater demand occurring in the region, we develop a parallel version of DYSOC wwhich incorporates strategy for consideration of expensive constraints in the process of exploring optimal objective function. DYSOC is developed based on DYCORS which is a variant of SRBF for problems with high dimensions. Three different scenarios are introduced, each describing a situation of pumping in different considerations, including maximizing extraction, prioritizing environmental protection or exploiting deep pumping. Results indicates that DYSOC is efficient compared to p-SRBF. The third part is dedicated to an exploration of the asynchronous paradigm of Parallel Surrogate-based Optimization algorithms using the PySOT toolbox in a calibration problem of groundwater flow and transport model. To efficiently incorporate the asynchronous version, we develop an early truncation strategy in each simulation so that for parameter sets which may lead to a bad solution, only partial simulation is spent to save the computing budget. Various coupling strategies have been tested on two different sites in the Umatilla problem. The asynchronous version algorithm with coupled Early Truncation Strategy shows a consistent and robust performance. In addition, in comparison with algorithms SCE-UA and APPSPACK, our coupled paradigm also has an better performance.