Analyzing and Designing for Deliberativeness in Online Policy Discussion

Abstract

Deliberation is a discussion-based practice that involves challenging a group of people to consider a policy issue by carefully weighing the diversity of perspectives held by members of the group. Deliberation theory expects that the practice contributes to an informed populous, well-reasoned decision-making, and mutual understanding among political adversaries. With such expectations, it is no wonder that deliberation theory and practices have been a source of inspiration when determining how to analyze an online policy discussion. However, applying concepts from deliberation to analyze an online policy discussion is not trivial. There is a wide variety of discussion-based protocols that are referred to as deliberation, that involve varying degrees of facilitation, decision-making, and dialogue. As a result of this variety, there is no clear standard method for analyzing how people engage with topics, perspectives and others during a deliberation (referred to as the state of a deliberation, or its "deliberativeness"). Additionally some assumptions about the group membership and task of a deliberation are less relevant in an online policy discussion setting. As evidence of how these conditions complicate the practice of studying deliberativeness, the dissertation contributes a systematic review of the common analytic decisions for studying deliberativeness in online policy discussion. A central argument in this dissertation is that the challenges involved with studying deliberativeness in online policy discussion can be addressed by tightly integrating analysis with system design research. In an analysis it is important to consider how the design of a system might influence what participants are likely to contribute during an online policy discussion. In the first of two case studies, I present results from an experiment that demonstrate how key analytic concerns, such as the topic coherence of a discussion in the face of disagreement, are influenced by design decisions about how to order existing comments in a discussion. Analysis concepts might also offer a valuable source of inspiration for online policy discussion system design. In the second case study, I present results from an experiment that evaluates a process for introducing newcomers to an online policy discussion, that was inspired by analysis concepts, specifically meta-talk about conflict. The residual benefit to the design "work" involved with operationalizing analysis concepts, like topic coherence and meta-talk, for an online context is that this work may help to narrow the gap between the theory and practice of deliberation and the study of deliberativeness in online policy discussion. In so doing, standards for operationalizing these concepts may emerge to ease the process of developing and evaluating theory about deliberativeness in online policy discussion. Insights about online policy discussion based on concepts from deliberation may also contribute back to the analysis and design of deliberation practices, whether in online, face-to-face, or even hybrid-online settings.