Scientific machine learning (SciML) combines models built from repeated linear algebraic operations (e.g. matrix multiplication, least squares) with scientific inductive biases, including differential equations, symmetries, and geometric structure. Thus, numerical linear algebra provides a natural language for SciML problems. In this dissertation, I show how numerical linear algebra can be used (1) as a tool to make existing machine learning models more efficient and (2) as a foundation for building new models. I present three algorithms that I developed: INTRIM, PruningAMR, and ContHutch++. INTRIM and PruningAMR illustrate (1), while ContHutch++ illustrates (2). INTRIM (IN-Training Iterative Model) Compression interleaves neural network training with repeated compression steps to remove excess capacity, reducing both training and inference time while maintaining accuracy. PruningAMR is an algorithm for efficiently visualizing implicit neural representations, which are neural networks that approximate functions of space and time. PruningAMR combines ideas from neural network compression and adaptive mesh refinement to construct adaptive-resolution visualizations that use substantially less memory than standard voxel representations. ContHutch++ estimates the trace of a trace-class integral operator by extending the matrix-based trace estimators Hutchinson’s estimator and Hutch++ to the operator setting with high-probability accuracy guarantees. I demonstrate each algorithm on multiple SciML problems, ranging from approximating molecular wavefunctions to visualizing time-varying CT scans to estimating the density ofstates of the Schr¨odinger operator.