A fundamental idea in machine learning (ML) has been the scaling of resources to increase model capability. More training iterations, hyperparameter tuning, model parameters, or data are generally beneficial. For a computational or memory resource, and a capability metric, there is a a Pareto frontier. Navigating this tradeoff curve, i.e. scaling resources to scale model capabilities, has powered much of the advancements in the modern ML age of the past 15 years. However, there is another way: the tradeoff curve can be improved. Scaling resources is not always viable, be it on edge devices or in data centers. My PhD has focused on this approach. Part I presents my work in post training compression. Given an already trained model, reduce the memory and/or compute footprint with minimal degradation. I propose methods in pruning and quantization. This problem formulation has become especially compelling with the prohibitive cost of creating LLMs. Part II presents my work in reducing hyperparameter tuning load. More tuning generally improves a model, but can be non-trivial. I use meta-learning to estimate optimal hyperparameters, and develop a more hyperparameter efficient statistical inference method. Part III goes beyond this tradeoff framework, which assumes a fixed objective. Improvements in recommender safety require reformulating the problem statement.