A widespread feature in the genomes of most bacteria and archaea is an array of clustered, regularly interspaced short palindromic repeats (CRISPRs) that, together with a group of CRISPR-associated (Cas) proteins, mediate immunity against infection by extrachromosomal agents such as phages. RNA molecules derived from the CRISPRs are predicted to guide the Cas proteins against foreign genetic elements with sequences matching the spacers. Here we found that bacteria can specifically silence plasmid DNA encoding an exported protein that induces envelope stress. Elimination of this plasmid DNA required base-complementarity between spacer DNA and the plasmid, and the participation of the Cas proteins. Of these, the CasC, D, and E proteins form a stable ternary complex and function as the catalytic core of the Cas system to process the pre-CRISPR RNA into the mature form. Additionally, CasE and several other Cas proteins possess DNase activity suggesting that these enzymes have a role in the effector function of the CRISPR-Cas system. We conclude that the Cas interference machinery can target DNA directly using CRISPR-derived small RNAs as guides and provides a general defense mechanism that is not limited to phage infection.