Lowshot Segmentation

Abstract

Learning from limited supervision has become an area of interest in machine learning because deep learning systems have demonstrated a dependence on large, labeled data sets which can be costly and sometimes unfeasible to obtain. A subset of this endeavor is the task of low-shot learning which is learning from a handful of labeled examples and has seen recent attention in image recognition. In this work we investigate the low-shot learning paradigm in the context of semantic segmentation, the challenges that it presents, and develop techniques for addressing these challenges. We first explore multiple ways in which limited labeling can occur in the task of semantic segmentation which is not a straight forward extension of its image recognition counter-part. Next we present novel techniques for increasing performance which include a form of learned data augmentation and incorporating inductive biases about local context to leverage unlabeled data. Finally we shift the experiments to a more natural and general setting in which the data naturally follows a heavy-tail distribution over the class frequency and size rather than artificially limiting labeled data for certain classes.