Soil contamination is often addressed by constructing new compost-based soils for urban agriculture. We examined these constructed soils to assess soil health across a diversity of farming sites across New York City, consulting with farmers to characterize their soils as “healthy” or “unhealthy” based on agricultural productivity. We analyzed these soils using the Cornell Comprehensive Assessment for Soil Health (CASH), characterized the soil microbiome, and conducted a taxonomic analysis of the soil invertebrate community. We found that farmers’ perceptions of soil health predicted differences in the bacterial community across all farms. The soil variables that consistently correlated with the biological communities in this study were respiration, pH, and magnesium. Many other soil variables affected the bacterial, fungal, and invertebrate communities individually; farmer perceptions of soil health were predictive of active carbon, organic matter, and predicted soil protein. Overall CASH ranked all sites highly regardless of farmer perceptions, indicating that standard soil health assessments may not fully capture key differences relevant to management of urban soils.