Understanding the complex relationship between selection and gene flow is a central goal in evolutionary biology. The Atlantic silverside (Menidia menidia), a small fish distributed across the steep latitudinal climate gradient of the North American Atlantic coast, is an excellent system to examine how selection and gene flow interact and shape patterns of genome evolution. Extensive research on this species has demonstrated a remarkable degree of local adaptation in multiple traits associated with the climatic gradient across its range. Leveraging the recently published Atlantic silverside reference genome, I investigated the genomic basis and architecture underlying adaptive divergence in this species. First, I conducted comparative linkage mapping, revealing suppressed recombination across massive chromosomal inversions that segregate between locally adapted populations. Next, I analyzed low-coverage whole genome sequencing data from four populations connected by varying levels of gene flow, finding striking signatures of differentiation only in specific genomic regions when divergent selection and gene flow co-occur. By integrating my results, I further discovered that genomic differentiation between populations is limited to inversions and putative centromeres, but the latter only show signatures of within-population selection, whereas inversions show strong signatures of divergent selection in the face of gene flow. Finally, through mapping of key adaptive traits, I confirmed that multiple strong QTL coincide with the major inversions that are under strong divergent selection. My research demonstrates how local adaptation is maintained despite homogenizing gene flow due to suppressed recombination between alternate arrangements of chromosomal inversions harboring adaptive loci.