We compute the effect of different isolated defects on the phonon transmission through carbon nanotubes, using an ab initio density functional approach. The problem of translational and rotational invariance fulfillment in the nonperiodic system is solved via a Lagrange-multiplier symmetrization technique. The need for an ab initio approach is illustrated for the case of phonon transmission through a nitrogen substitutional impurity, for which no reliable empirical interatomic potentials exist. This opens an avenue for the accurate parameterfree study of phonon transport through general systems with arbitrary composition and structure, without any need for semiempirical potential descriptions.