We develop a phonon-magnon interaction model for the two-dimensional (2D) honeycomb ferromagnet and calculate the band structure of the hybrid phonon-magnon excitation. When the phonon (either acoustic or optical) energy overlaps with the magnon energy, the degeneracies at the crossing points are lifted by the phonon-magnon interaction, leading to topological band inversion. We show that the Berry curvatures are nonzero around the phonon-magnon band crossing areas but vanish throughout the rest of the Brillouin zone. We calculate the thermal Hall conductance enabled by the phonon-magnon interaction, which is also an indicator of nontrivial topological properties. Moreover, we show that band topologies can be manipulated by varying spin exchange coupling strengths and Dzyaloshinskii-Moriya interactions (DMI) strengths.