Differential Dynamic Programming (DDP) and stagewise Newton's method are both quadratically convergent algorithms for solving discrete time optimal control problems. Although these two algorithms share many theoretical similarities, they demonstrate significantly different numerical performance. In this paper, we will compare and analyze these two algorithms in detail and derive another quadratic- ally convergent algorithm which is a combination of the DDP algorithm and Newton's method. This new second-order algorithm plays a key role in the explanation of the numerical differences between the DDP algorithm and Newton's method. The detailed algorithmic and structural differences for these three algorithms and their impact on numerical performance will be discussed and explored. Two test problems with various dimensions solved by these three algorithms will be presented. One nonlinear test problem demonstrates that the DDP algorithm can be as much as 28 times faster than the stagewise Newton's method. The numerical comparsion indicates that the DDP algorithm is numerically superior to the stagewise Newton's method.