Experimental and theoretical work has suggested that persistent activity in short-term memory circuits is supported by excitatory recurrent connections. However, the organization of these connections remains difficult to accurately specify due to the typically low-dimensional nature of the recorded activity. To address this problem we use two-photon calcium imaging in the oculomotor integrator of larval zebrafish to study network activity under several different experimental conditions. By studying the network under multiple behavioral contexts, we have found that, unlike traditional "line attractor" models of connectivity, the network can support activity that persists for tens of seconds along two distinct dimensions which are accessed in a contextdependent manner. When put into a modeling framework, these dynamics put important constraints on the distribution of potential connectivity structures. To further refine our estimates of connectivity, we have used electrical microstimulations to explore the dynamics along an increasing number of state-space dimensions. We find that the observed dynamics are most consistent with a picture in which connections are strongest between neighboring neurons.