Aligning natural language processing models with stakeholder values is critical to managing the social biases that a model may learn from data. How to select and use context so that the resulting models are sensitive to these values is still an open question. I present three studies in which linguistic social context, or context documents, are leveraged for natural language processing tasks using base documents from social data (i.e., Wikipedia and conversation data). The context documents are produced by the members of the community as part of pre-existing processes. The context documents also enable an approach that trades the burden of precise annotation for noisier but value-sensitive information in those documents. I use techniques from semi-supervised learning and distant supervision to incorporate the information extracted from context documents into several inference tasks.