Structural and functional brain connectivity can be estimated from neuroimaging data through various techniques. Structural connectivity — patterns of anatomical white matter connections — is estimated using diffusion MRI (dMRI) whereas functional connectivity — patterns of neural coactivation — can be estimated using functional MRI (fMRI). The relationship between structural connections and functional coactivation varies with individual characteristics such as age or cognitive ability, and in disease or injury. Understanding the structure–function relationship is challenging, especially given a lack of consensus for how either structural or functional connectivity should be estimated from dMRI or fMRI. Opinions vary on issues from how to reduce spatial complexity to how to account for methodological biases and confounds or physiological artifacts, and different choices affect the findings. To align the diverse set of connectivity estimates, we set out to consider them as complementary views of the same underlying network. This network, when observed through each set of measurement and processing choices, presents as the set of observed connectivity estimates. This alignment can both allow translation between techniques and provide a unified connectivity estimate. We created a many-armed encoding model, dubbed the Krakencoder, as a set of autoencoders with a shared latent representation. This model decomposes data from each connectivity flavor into components that are then aligned with those from other flavors and modalities. The Krakencoder was able to translate connectivity patterns estimated from the same individual using different spatial resolutions and processing strategies. When predicting functional connectivity from structural, our model significantly outperformed other models from literature in both reconstruction accuracy and individual prediction identifiability. Furthermore, our unified low-dimensional latent representation better reflected genetic similarity between participants than the observed high-dimensional connectivity data and was more predictive of phenotypic features such as cognitive performance. The Krakencoder can be applied, without retraining, to new out-of-distribution data while still preserving inter-individual differences in the connectome predictions and familial relationships in the latent representations. To further improve generalizability, an expanded model was trained on a multi-site, multi-study cohort of healthy subjects across the lifespan (age 8-100), using adversarial learning to remove confounding site and study-specific information from the latent representation. The Krakencoder is a notable leap forward in capturing the relationship between multimodal brain connectomes in an individualized, behaviorally and demographically relevant way.