This dissertation deals with the transition planning and operations scheduling of smart energy systems using optimization and machine learning, aiming to address challenges in accommodating renewable energy penetration. This dissertation covers four aspects, including the development of decarbonization pathways for regional energy transition, incentive policy design for the adoption of renewable energy, waste-to-energy supply chain optimization, and power systems operations with intermittent renewable energy. The first research aspect includes three distinct research projects. In the first related project, we propose a bottom-up optimization modeling framework for energy transitions, which bridges the power sector and other energy sectors represented by space heating. In the second related project, we develop a bottom-up data-driven multistage adaptive robust optimization framework to investigate energy transition pathways under uncertainty. We apply affine decision rules to overcome the computational intractability. Machine learning techniques are applied for constructing data-driven uncertainty sets to avoid over-conservation. In the third related project, we propose a multi-scale bottom-up renewable electricity transition optimization framework to simultaneously address yearly systems design decisions and hourly operations decisions. A novel machine learning-based approach is used to reduce computational demand. For the second research aim, we propose an optimization framework based on single-leader-multiple-follower Stackelberg game to promote bioenergy generation. We formulate a bi-criterion mixed-integer bilevel fractional programming problem, and a tailored global optimization algorithm integrating a parametric algorithm and a projection-based reformulation and decomposition algorithm is developed for solving. For the third research aim, we develop a life-cycle optimization framework for poultry waste supply chain considering fast and slow pyrolysis technologies, which is formulated into multiobjective mixed-integer fractional linear programming. The fourth research aspect includes two distinct projects. In the first one, we propose a machine learning-based two-stage adaptive robust optimization framework with data-driven disjunctive uncertainty sets for volatile renewable generation to address renewable energy-induced disjunctive uncertainties in power systems operations. To facilitate the solution process, a tailored decomposition-based optimization algorithm is developed. In the second project, we develop an optimal power flow framework to address the operations of the future zero-carbon grid, aiming to involve hydrogen-based long-term energy storage to tackle the seasonal load shedding and transmission line congestion issues of deeply decarbonized power systems while capturing the variety of future climate scenarios, the topology, and operational requirements of the power systems.