Effective representation of the sensory world is a complex task because the statistics of sensory events fluctuate over many timescales. Here we study how the brain uses recent stimulus history to alter its representations of sensory information in primate visual cortex, with a focus on the role of inhibition and whether adaptation acts at multiple sites. Adaptation to gratings induced changes in gain, bandwidth, and orientation preference in both excitatory and inhibitory cell types. We find that tuned inhibitory cells adapt in a manner that is distinct from their excitatory counterparts - effects that are consistent with the notion of adaptation-induced disinhibition of cortical interactions. We further show that some of the diversity of effects present at the population level reflects location on the orientation map: cells in pinwheel centers have generally larger adaptation-induced shifts. Moreover, a subset of untuned neurons become tuned after adaptation. Next, we find that adaptation effects in extrastriate area V2 go beyond effects inherited from striate area V1: specifically, attractive shifts to stimuli that are nearly orthogonal to the preferred direction are much larger in V2 compared to V1. The prominence of these attractive shifts correlates with classical perceptual after-effects. More broadly, this finding shows that adaptation modulates sensory processing at multiple cortical levels. Finally, modeling demonstrates that tuned inhibition and pre-synaptic adaptation are required to account for the observed magnitude, direction, and diversity of effects.