Realizing the Human Potential of Program Synthesis and Crowdsourcing
Computational methods have great potential to solve a wide range of problems, from those tightly connected to computing to those farther afield in human domains. In fact, applying computational techniques to human-centered problems has always been one of the aims of computing (Shannon 1950, Turing 1950). Yet, for many techniques, their impact on human-centered problems has been rather limited. We need to bridge both an interaction and an implementation gap in order to realize the human potential of existing computational techniques. The interaction gap refers to the difference between how systems take input and present output and the practices that humans are used to. This gap is similar to how Norman describes the Gulfs of Evaluation and Execution (Norman 1986, Norman & Draper 1986). The implementation gap refers to the difference between how a computational technique was originally designed to be used and its use in solving new human-centered problems. In this dissertation, I argue that both of these gaps can be bridged through the use of cognitive models; specifically, that cognitive models of human practice enable the successful application of computational techniques to human-centered problems. This dissertation describes three instantiations of this general approach. The first project considers if and how cognitive models play a role in the way designers bridge the implementation gap in existing systems. In particular, I use an existing cognitive process of writing to fully analyze and characterize how crowdsourcing has been applied to generating original written content. This work describes and elucidates the decisions designers make when building crowdsourcing systems for writing. I find that designers both implicitly and explicitly use cognitive approaches when building systems. The second piece of work considers using cognitive research from math education to identify student misconceptions in K-8 mathematics using program synthesis. My work addresses the interaction gap through evaluating the accuracy of programs produced by an existing misconception analysis system and developing a method of conveying them visually to an educator. Finally, the third project considers both the interaction and implementation gaps. It applies program synthesis to provide real-time feedback in introductory computer science. I present a formative study of the practices of teaching assistants in their office hours to provide a cognitive basis. Then I adapt that basis to frame feedback as a query to an existing program synthesis system, resolving the implementation gap. In addition, this work addresses the interaction gap by discussing the challenge of conveying synthesized output in real-time to users. Taken together, the work presented in this dissertation suggests that computing can achieve its human potential through the use of cognitive models.