Representation Without Representationalism

Abstract

The idea that representations not only describe, but also help shape the world is being explored both empirically and theoretically in an increasing number of academic disciplines ranging from anthropology to quantum mechanics. Insights emerging from such research question representationalism: the belief that representations simply describe the represented. Ideas and arguments around the nature of representation are particularly relevant for computer science because computers are representational technologies: in order to be useful, they must represent relevant aspects of the world. In my PhD research, I have taken up the challenge of exploring the implications of these ideas for computational approaches. Grounded in affective computing and ubiquitous computing, my research was guided by two core questions united by a focus on the opportunities and implications brought about by taking seriously the idea that representations also shape reality. The first examines how to derive computational representations differently by engaging this idea in technical practice. Two projects provide two case studies on different representations using physiological sensor data: one on basic visualizations of the data; the other, focuses on a more complex form of representation: training classifiers of emotion from the data using machine learning. Building on these projects, the second question examines how the shift in the way we understand representations changes the main practices of constructing interactive systems. To this end, I have designed, built, and evaluated an interactive, mobile system leveraging sensor based statistical classification; the system is designed to help its users experience and understand their emotions.