Riparian restoration and protection provide key benefits in terms of both clean water and resilient communities. Unfortunately, key knowledge gaps exist in terms of understanding the current state and effectiveness of riparian corridors in New York State. The primary objective of this study was to develop a method to delineate buffer vegetation and quantify the spatiotemporal patterns of buffer vegetation variation over time. We tested our method in a sub-watershed of the Hudson River-Stockport Creek Watershed. With an overall accuracy of classifying vegetation and non-vegetation regions within aerial photos above 97%, we were able to confidently determine spatial trends in buffer vegetation for 2006 and 2015. We tested four different buffer sizes and found that there were more areas of vegetation than non-vegetation regardless of size. Hotspot analysis reveals that 15–22% of buffer areas have clusters of vegetation degradation within each year, 30–40% of buffer areas are persistently degraded across the time span studied. Ultimately, our approach can enable stakeholders, managers and community groups to not only rapidly delineate buffer vegetation from high spatial resolution aerial images, but also enable them to assess the spatiotemporal patterns of vegetation variations.