Complex multiparameter models such as in climate science, economics, systems biology, materials science, neural networks and machine learning have a large-dimensional space of undetermined parameters as well as a large-dimensional space of predicted data. These high-dimensional spaces of inputs and outputs pose many challenges. Recent work with a diversity of nonlinear predictive models, microscopic models in physics, and analysis of large datasets, has led to important insights. In particular, it was shown that nonlinear fits to data in a variety of multiparameter models largely rely on only a few stiff directions in parameter space. Chapter 2 explores a qualitative basis for this compression of parameter space using a model nonlinear system with two time scales. A systematic separation of scales is shown to correspond to an increasing insensitivity of parameter space directions that only affect the fast dynamics. Chapter 3 shows with the help of microscopic physics models that emergent theories in physics also rely on a sloppy compression of the parameter space where macroscopically relevant variables form the stiff directions. Lastly, in chapter 4, we will learn that the data space of historical daily stock returns of US public companies has an emergent simplex structure that makes it amenable to a low-dimensional representation. This leads to insights into the performance of various business sectors, the decomposition of firms into emergent sectors, and the evolution of firm characteristics in time.