Self-assembly is a powerful technique for manufacturing at small length scales, exploiting physics-driven processes to access size regimes where top-down manufacturing is slow and costly. The assembly of colloidal nanoparticles creates structures that can be nanometers to millimeters across, and the same structure can be observed in diverse materials, allowing for designer photonic, plasmonic, and catalytic properties. However, in order for self-assembly to be a viable manufacturing technique, it must be controlled to target desirable structures and high yield. This thesis investigates how confinement and clustering affect the structure of colloids. Colloidal clusters describe any system in which the assembly is limited by an external field, a confining droplet, or by self-limiting growth. In particular, the phenomenon of Magic-Size Clusters (MSCs) describes how certain cluster sizes are particularly stable, making them good targets for designing highly monodisperse synthesis and assembly processes. Monte Carlo and molecular dynamics simulations of particle self-assembly are used to investigate how anisotropy, compressibility, and size polydispersity influence clustering and change or eliminate MSCs. In a system of anisotropic tetrahedral particles, confinement can be used to template the growth of geometrically matched structures and overcome kinetic barriers to crystallization, illuminating paths to manufacture high-crystallinity, low-defect colloidal crystals. In smaller clusters, tuning the softness of colloidal particles changes the driving forces towards assembly and thus the relevant, highly stable “magic” sizes, while size polydisperse particles exhibit a decreased significance of magic sizes but higher overall stability. Finally, applying this work to a system of silica nanoparticles used for imaging and medical applications shows that self-limited growth yields ultrasmall MSCs. Experimental and simulated assembly demonstrates that targeting magic sizes can be used to manufacture highly uniform nanoparticles in a robust, scalable manner. This work will guide future research into colloidal assemblies and gives insight into the fundamental mechanisms of structure formation in clusters.