Large-scale AI systems have achieved striking results across vision, language, and reasoning, yet critical gaps remain. Learning from scratch typically requires millions of labeled examples or environment interactions. Models that perform well within their training distribution often fail when inputs shift in small but systematic ways. And the computational cost of generation and inference continues to grow, as longer reasoning chains and larger candidate pools multiply expense with each step toward harder problems. These gaps are not merely engineering inconveniences---they determine whether AI systems can be trusted, deployed broadly, and pushed to problems where brute-force scaling falls short. Programs are how humans have long expressed precise, executable knowledge about the world---from scientific simulations to planning algorithms to the software running everyday infrastructure. They compose, they can be verified by running them, and the same notation that describes one problem often describes a family of related ones. Large language models, trained on vast corpora of human-written code, have already internalized this language. This thesis asks what happens when AI systems reason and represent in this same medium: not as a special case, but as a natural alignment between how humans structure solutions and how machines can learn to produce them. Across five projects spanning program synthesis, world modeling, code repair, video prediction, and efficient reasoning, we find that this alignment yields order-of-magnitude improvements in sample efficiency, compute efficiency, and generalization that purely statistical scaling does not. ROAP learns programs jointly with perception from raw sensory data by combining regularization, overparameterization, and amortization; programs trained on digits 0--5 achieve 99.8% accuracy on digits 6--9, compared to at most 17% for purely neural baselines---a $10\times$ reduction in error. WorldCoder represents world knowledge as executable Python code: an LLM generates and refines transition and reward functions under consistency constraints, producing an accurate and verifiable world model from roughly 50 environment interactions, versus over one million for leading neural alternatives, while supporting zero-shot transfer to new goals. REx treats iterative LLM-based code repair as an arm-acquiring bandit problem and uses Thompson Sampling to dynamically balance refinement against exploration across a growing tree of candidates; this achieves equivalent or better performance than prior search strategies using 20--80% fewer LLM calls, and sets a new state of the art on loop invariant synthesis. ProgGen applies LLM-based program synthesis to video prediction, decomposing visual dynamics into perception, transition, and rendering programs that are neuro-symbolic and human-interpretable; trained on only 10 videos, it outperforms diffusion models trained on up to three million videos in out-of-distribution settings, while enabling counterfactual reasoning and editability. RWR reduces cost within a single reasoning rollout by retaining only the first tokens (problem context) and most recent tokens (active reasoning), discarding the intervening chain that contributes little to future predictions; this position-based rule reduces attention by up to $4\times$ and memory by $2\times$, preserving accuracy down to 50% of the original KV-cache budget. Together, these results show that programs---as a substrate---carry enough structure to make reliable and efficient AI within reach at a fraction of the data and compute that unstructured learning requires.