Penalized splines approach has very important applications in statistics. The idea is to fit the unknown regression mean using a high-dimensional spline basis, subject to penalization on the roughness. Such an approach avoids the stringency of a parametric model and enables a considerable reduction in computational cost without a loss of statistical precision. Moreover, the idea can also be connected with ridge regression and mixed models, thus allowing more flexible handling of longitudinal and spatial correlation. This thesis focuses on nonparametric and semiparametric estimation and inference using penalized splines. First, we consider the penalized splines approach proposed by Eilers and Marx (1996), which is also called P-splines approach. We derive its asymptotic property when the number of spline basis increases as the sample size does. For both the univariate model and the additive models, we establish the asymptotic distribution of the estimators and give simple expressions for the asymptotic mean and variance. Such an asymptotic theory allows P-splines estimators to be compared theoretically with other nonparametric estimators and offers guidance for practitioners when considering the choice of the penalty and basis functions. Next, we turn to the global inferential problems for functional data. We model the population mean function using polynomial splines. By utilizing the mixed model based penalized splines approach, we treat some of the spline coefficients as random effects with a single variance component and relate hy- potheses of interest to tests with this variance component being zero. To take into account the dependent structure or within subject correlation, we propose a pseudo likelihood test statistic and derive its null distribution. This work extends existing results on pseudo likelihood by allowing the use of nonparametric smoothing (usually with a slower convergence rate). Its effectiveness is demonstrated via simulations and the empirical application from the Sleep Health Heart Study.