In interactive machine learning systems (IMLSs), such as search engines, social networks, and e-commerce sites, machine learning algorithms and user interfaces are inseparably linked. My thesis demonstrates that improving the accuracy of the machine learning algorithm in such systems is not only a question of the algorithm itself, but also a question of the user interface that directly affects the properties of feedback data the machine learner receives. To this end, this thesis introduces the concept of feedback-enhancing interface design as an alternate and complementary pathway to better machine learning performance. As I will show in this thesis, feedback-enhancing interfaces allow us to effectively shape the quantity as well as the quality of the obtained feedback data, all while maintaining usability and user experience.