Nowadays flight travel becomes a common way for people for work or leisure. With more frequent flight travels, lots of manual instructions by the ground tower and by ground crew members are needed. More resources and costs are accumulated, making up a big budget for the airplane managers to consider. Thus, a methodology of automatically guiding the airplanes to taxi based on the guidance gestures from the ground crew member is proposed. Many method- ologies are used for human actions segmentation and change points detection of the human movements. Most of the algorithms focus on the single type of human action, which cannot generalize to the everyday human activities. This thesis presents a methodology that combine both the actions segmentation and recognition parts, and direct the airplanes based on the action recognition re- sults.