Jonathan P. Chang
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Incivility remains a major challenge for online discussion platforms, to such an extent that even conversations between well-intentioned users can often derail into uncivil behavior. Traditionally, platforms have relied on moderators to---with or without algorithmic assistance---take corrective actions such as removing comments or banning users. In this work we propose a complementary paradigm that directly empowers users by proactively enhancing their awareness about existing tension in the conversation they are engaging in and actively guides them as they are drafting their replies to avoid further escalation.
As a proof of concept for this paradigm, we design an algorithmic tool that provides such proactive information directly to users, and conduct a user study in a popular discussion platform. Through a mixed methods approach combining surveys with a randomized controlled experiment, we uncover qualitative and quantitative insights regarding how the participants utilize and react to this information. Most participants report finding this proactive paradigm valuable, noting that it helps them to identify tension that they may have otherwise missed and prompts them to further reflect on their own replies and to revise them. These effects are corroborated by a comparison of how the participants draft their reply when our tool warns them that their conversation is at risk of derailing into uncivil behavior versus in a control condition where the tool is disabled.These preliminary findings highlight the potential of this user-centered paradigm and point to concrete directions for future implementations.
Supplemental Material
- Ofer Arazy, Lisa Yeo, and Oded Nov. 2013. Stay on the Wikipedia Task : When Task-related Disagreements Slip Into Personal and Procedural Conflicts. J. Assoc. Inf. Sci. Technol. , Vol. 64, 8 (Aug. 2013).Google Scholar
- Zahra Ashktorab and Jessica Vitak. 2016. Designing Cyberbullying Mitigation and Prevention Solutions through Participatory Design With Teenagers. In Proceedings of CHI.Google ScholarDigital Library
- Jiajun Bao, Junjie Wu, Yiming Zhang, Eshwar Chandrasekharan, and David Jurgens. 2021. Conversations Gone Alright : Quantifying and Predicting Prosocial Outcomes in Online Conversations. In Proceedings of WWW.Google ScholarDigital Library
- Matt Billings and Leon A. Watts. 2010. Understanding Dispute Resolution Online: Using Text to Reflect Personal and Substantive Issues in Conflict. In Proceedings of CHI.Google Scholar
- Paolo Bory. 2019. Deep New: The Shifting Narratives of Artificial Intelligence from Deep Blue to AlphaGo. Convergence, Vol. 25, 4 (Aug. 2019).Google Scholar
- Johanna Brewer, Morgan Romine, and T. L. Taylor. 2020. Inclusion at Scale : Deploying a Community-Driven Moderation Intervention on Twitch. In Proceedings of DIS.Google Scholar
- Barry Brown, Stuart Reeves, and Scott Sherwood. 2011. Into the Wild: Challenges and Opportunities for Field Trial Methods. In Proceedings of CHI.Google ScholarDigital Library
- Penelope Brown and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge University Press.Google Scholar
- Jie Cai and Donghee Yvette Wohn. 2019. What Are Effective Strategies of Handling Harassment on Twitch ? Users ' Perspectives. In Proceedings of CSCW.Google Scholar
- Jie Cai, Donghee Yvette Wohn, and Mashael Almoqbel. 2021. Moderation Visibility : Mapping the Strategies of Volunteer Moderators in Live Streaming Micro Communities. In Proceedings of IMX.Google ScholarDigital Library
- Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You Can 't Stay Here : The Efficacy of Reddit 's 2015 Ban Examined Through Hate Speech. In Proceedings of CSCW.Google Scholar
- Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet 's Hidden Rules : An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. In Proceedings of CSCW.Google Scholar
- Jonathan P Chang, Justin Cheng, and Cristian Danescu-Niculescu-Mizil. 2020. Don't Let Me Be Misunderstood :Comparing Intentions and Perceptions in Online Discussions. In Proceedings of WWW.Google ScholarDigital Library
- Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019. Trouble on the Horizon : Forecasting the Derailment of Online Conversations as They Develop. In Proceedings of EMNLP.Google Scholar
- Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll : Causes of Trolling Behavior in Online Discussions. In Proceedings of CSCW.Google ScholarDigital Library
- Emily I. M. Collins, Anna L. Cox, Jon Bird, and Daniel Harrison. 2014. Social Networking Use and RescueTime : The Issue of Engagement. In Proceedings of UbiComp Adjunct.Google ScholarDigital Library
- Sunny Consolvo, David W. McDonald, Tammy Toscos, Mike Y. Chen, Jon Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand, Ryan Libby, Ian Smith, and James A. Landay. 2008. Activity Sensing in the Wild: A Field Trial of Ubifit Garden. In Proceedings of CHI.Google ScholarDigital Library
- Kate Crawford and Tarleton Gillespie. 2016. What Is a Flag for? Social Media Reporting Tools and the Vocabulary of Complaint. New Media & Society, Vol. 18, 3 (March 2016).Google ScholarCross Ref
- Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of ACL.Google Scholar
- Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of ICWSM.Google ScholarCross Ref
- Julian Dibbell. 2005. A Rape in Cyberspace. The Village Voice (Oct. 2005).Google Scholar
- Bryan Dosono and Bryan Semaan. 2019. Moderation Practices as Emotional Labor in Sustaining Online Communities : The Case of AAPI Identity Work on Reddit. In Proceedings of CHI.Google ScholarDigital Library
- Natasha Duarte and Emma Llansó. 2018. Mixed Messages ? The Limits of Automated Social Media Content Analysis. In Proceedings of FAccT.Google Scholar
- R. Stuart Geiger. 2016. Bot-Based Collective Blocklists in Twitter : The Counterpublic Moderation of Harassment in a Networked Public Space. Information, Communication & Society , Vol. 19, 6 (June 2016).Google ScholarCross Ref
- R. Stuart Geiger and David Ribes. 2010. The Work of Sustaining Order in Wikipedia: The Banning of a Vandal. In Proceedings of CSCW.Google Scholar
- Alex Georgakopoulou, Stefan Iversen, and Carsten Stage. 2020. Making Memes Count : Platformed Rallying on Reddit. In Quantified Storytelling : A Narrative Analysis of Metrics on Social Media, , Alex Georgakopoulou, Stefan Iversen, and Carsten Stage (Eds.). Springer International Publishing, Cham.Google Scholar
- Sarah A. Gilbert. 2020. "I Run the World's Largest Historical Outreach Project and It's on a Cesspool of a Website." Moderating a Public Scholarship Site on Reddit : A Case Study of r/AskHistorians. In Proceedings of CSCW.Google ScholarDigital Library
- Tarleton Gillespie. 2018. Custodians of the Internet : Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press, New Haven.Google Scholar
- Tarleton Gillespie. 2020. Content Moderation, AI, and the Question of Scale:. Big Data & Society (July 2020).Google Scholar
- Tarleton Gillespie, Patricia Aufderheide, Elinor Carmi, Ysabel Gerrard, Robert Gorwa, Ariadna Matamoros-Fernández, Sarah T. Roberts, Aram Sinnreich, and Sarah Myers West. 2020. Expanding the Debate about Content Moderation: Scholarly Research Agendas for the Coming Policy Debates. Internet Policy Review, Vol. 9, 4 (Oct. 2020).Google ScholarCross Ref
- Joseph K. Goodman, Cynthia E. Cryder, and Amar Cheema. 2013. Data Collection in a Flat World : The Strengths and Weaknesses of Mechanical Turk Samples. Journal of Behavioral Decision Making , Vol. 26, 3 (2013).Google ScholarCross Ref
- Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic Content Moderation: Technical and Political Challenges in the Automation of Platform Governance. Big Data & Society, Vol. 7, 1 (Jan. 2020).Google ScholarCross Ref
- James Grimmelmann. 2015. The Virtues of Moderation. Yale Journal of Law and Technology , Vol. 17, 1 (Sept. 2015).Google Scholar
- Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before Name-Calling : Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation. In Proceedings of NAACL.Google ScholarCross Ref
- Aaron Halfaker, Bryan Song, D. Alex Stuart, Aniket Kittur, and John Riedl. 2011. NICE : Social Translucence Through UI Intervention. In Proceedings of WikiSym.Google ScholarDigital Library
- Francis Heylighen and Jean-Marc Dewaele. 1999. Formality of Language : Definition, Measurement and Behavioral Determinants. Technical Report. Center "Leo Apostel", Free University of Brussels.Google Scholar
- Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan, and Kathy McKeown. 2017. Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum. In Proceedings of the 4th Workshop on Argument Mining.Google ScholarCross Ref
- Eric Holgate, Isabel Cachola, Daniel Preotiuc-Pietro, and Junyi Jessy Li. 2018. Why Swear ? Analyzing and Inferring the Intentions of Vulgar Expressions. In Proceeding of EMNLP.Google Scholar
- Karen Huang, Michael Yeomans, Alison Wood Brooks, Julia Minson, and Francesca Gino. 2017. It Doesn't Hurt to Ask: Question-asking Increases Liking. Journal of Personality and Social Psychology, Vol. 113, 3 (Sept. 2017).Google ScholarCross Ref
- Krithika Jagannath, Katie Salen, and Petr Slovàk. 2020. "(We ) Can Talk It Out. ..": Designing for Promoting Conflict-Resolution Skills in Youth on a Moderated Minecraft Server. In Proceedings of CSCW.Google ScholarDigital Library
- Shagun Jhaver, Darren Scott Appling, Eric Gilbert, and Amy Bruckman. 2019a. “Did You Suspect the Post Would Be Removed ?”: Understanding User Reactions to Content Removals on Reddit. In Proceedings of CSCW.Google ScholarDigital Library
- Shagun Jhaver, Iris Birman, Eric Gilbert, and Amy Bruckman. 2019b. Human-Machine Collaboration for Content Regulation : The Case of Reddit Automoderator. ACM Transactions on Computer-Human Interaction, Vol. 26, 5 (July 2019).Google ScholarDigital Library
- Shagun Jhaver, Larry Chan, and Amy Bruckman. 2018a. The View from the Other Side: The Border between Controversial Speech and Harassment on Kotaku in Action. First Monday, Vol. 23, 2 (Feb. 2018).Google ScholarCross Ref
- Shagun Jhaver, Sucheta Ghoshal, Amy Bruckman, and Eric Gilbert. 2018b. Online Harassment and Content Moderation : The Case of Blocklists. ACM Transactions on Computer-Human Interaction, Vol. 25, 2 (March 2018).Google ScholarDigital Library
- Shagun Jhaver, Pranil Vora, and Amy Bruckman. 2017. Designing for Civil Conversations : Lessons Learned from ChangeMyView. Technical Report. Georgia Institute of Technology.Google Scholar
- Christian Katzenbach. 2021. “AI Will Fix This” textendash The Technical, Discursive, and Political Turn to AI in Governing Communication. Big Data & Society, Vol. 8, 2 (July 2021).Google ScholarCross Ref
- Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating Behavior in Online Communities. In Building Successful Online Communities : Evidence-Based Social Design, , Paul Resnick and Robert Kraut (Eds.). MIT Press.Google Scholar
- Aniket Kittur and Robert E. Kraut. 2008. Harnessing the Wisdom of Crowds in Wikipedia : Quality Through Coordination. In Proceedings of CSCW.Google Scholar
- Geza Kovacs, Zhengxuan Wu, and Michael S. Bernstein. 2018. Rotating Online Behavior Change Interventions Increases Effectiveness But Also Increases Attrition. In Proceedings of CSCW.Google Scholar
- Paul Krebs, James O. Prochaska, and Joseph S. Rossi. 2010. A Meta-Analysis of Computer-Tailored Interventions for Health Behavior Change. Preventive Medicine, Vol. 51, 3 (Sept. 2010).Google ScholarCross Ref
- Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012a. Supporting Reflective Public Thought with Considerit. In Proceedings of CSCW.Google ScholarDigital Library
- Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012b. Is This What You Meant?: Promoting Listening on the Web with Reflect. In Proceedings of CHI.Google ScholarDigital Library
- Srijan Kumar, Justin Cheng, Jure Leskovec, and V.S. Subrahmanian. 2017. An Army of Me : Sockpuppets in Online Discussion Communities. In Proceedings of WWW.Google Scholar
- Srijan Kumar, William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2018. Community Interaction and Conflict on the Web. In Proceedings of WWW.Google ScholarDigital Library
- Robin T. Lakoff. 1973. The Logic of Politeness : Minding Your P 's and Q 's. Chicago Linguistic Society.Google Scholar
- Cliff Lampe and Paul Resnick. 2004. Slash(Dot) and Burn: Distributed Moderation in a Large Online Conversation Space. In Proceedings of CHI.Google ScholarDigital Library
- Ping Liu, Joshua Guberman, Libby Hemphill, and Aron Culotta. 2018. Forecasting the Presence and Intensity of Hostility on Instagram Using Linguistic and Social Features. In Proceedings of ICWSM.Google ScholarCross Ref
- Claudia (Claudia Wai Yu) Lo. 2018. When All You Have Is a Banhammer : The Social and Communicative Work of Volunteer Moderators. Thesis. Massachusetts Institute of Technology.Google Scholar
- Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann. 2011. Design Lessons from the Fastest Q &A Site in the West. In Proceedings of CHI.Google ScholarDigital Library
- Aske Mottelson and Kasper Hornbæk. 2017. Virtual Reality Studies Outside the Laboratory. In Proceedings of VRST.Google ScholarDigital Library
- Daniel J. O'Keefe. 1997. Standpoint Explicitness and Persuasive Effect : A Meta-Analytic Review of the Effects of Varying Conclusion Articulation in Persuasive Messages. Argumentation and Advocacy , Vol. 34, 1 (June 1997).Google ScholarCross Ref
- Daniel J. O'Keefe. 1998. Justification Explicitness and Persuasive Effect : A Meta-Analytic Review of the Effects of Varying Support Articulation in Persuasive Messages. Argumentation and Advocacy , Vol. 35, 2 (Sept. 1998).Google ScholarCross Ref
- James W. Pennebaker, Cindy K. Chung, Joey Frazee, Gary M. Lavergne, and David I. Beaver. 2014. When Small Words Foretell Academic Success : The Case of College Admissions Essays. PLOS ONE (2014).Google Scholar
- Katharina Reinecke and Krzysztof Z. Gajos. 2015. LabintheWild : Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of CSCW.Google Scholar
- Annika Richterich. 2014. `Karma, Precious Karma !' Karmawhoring on Reddit and the Front Page 's Econometrisation. Journal of Peer Production , Vol. 4, 1 (2014).Google Scholar
- Sarah T Roberts. 2014. Behind the Screen: The Hidden Digital Labor of Commercial Content Moderation. Ph.,D. Dissertation. University of Illinois at Urbana-Champagne.Google Scholar
- Joseph Seering. 2020. Reconsidering Self-Moderation : The Role of Research in Supporting Community-Based Models for Online Content Moderation. In Proceedings of CSCW.Google ScholarDigital Library
- Joseph Seering, Tianmi Fang, Luca Damasco, Mianhong 'Cherie' Chen, Likang Sun, and Geoff Kaufman. 2019a. Designing User Interface Elements to Improve the Quality and Civility of Discourse in Online Commenting Behaviors. In Proceedings of CHI.Google ScholarDigital Library
- Joseph Seering, Geoff Kaufman, and Stevie Chancellor. 2020. Metaphors in Moderation. New Media & Society (Oct. 2020).Google ScholarCross Ref
- Joseph Seering, Robert Kraut, and Laura Dabbish. 2017. Shaping Pro and Anti-Social Behavior on Twitch Through Moderation and Example-Setting. In Proceedings of CSCW.Google ScholarDigital Library
- Joseph Seering, Tony Wang, Jina Yoon, and Geoff Kaufman. 2019b. Moderator Engagement and Community Development in the Age of Algorithms. New Media & Society, Vol. 21, 7 (July 2019).Google ScholarCross Ref
- Kumar Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, and Chenhao Tan. 2019. Content Removal as a Moderation Strategy : Compliance and Other Outcomes in the ChangeMyView Community. In Proceedings of CSCW.Google ScholarDigital Library
- Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu, and Lillian Lee. 2016. Winning Arguments : Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions. In Proceedings of WWW.Google ScholarDigital Library
- Samuel Hardman Taylor, Dominic DiFranzo, Yoon Hyung Choi, Shruti Sannon, and Natalya N. Bazarova. 2019. Accountability and Empathy by Design : Encouraging Bystander Intervention to Cyberbullying on Social Media. In Proceedings of CSCW.Google Scholar
- Kal Turnbull. 2018. “That 's Bullshit ” textendash Rude Enough for Removal ? A Multi-Mod Perspective.Google Scholar
- Zhongyu Wei, Yang Liu, and Yi Li. 2016. Is This Post Persuasive ? Ranking Argumentative Comments in the Online Forum. In Proceedings of ACL.Google ScholarCross Ref
- Galen Weld, Amy X. Zhang, and Tim Althoff. 2022. What Makes Online Communities 'Better '? Measuring Values, Consensus, and Conflict across Thousands of Subreddits. In Proceedings of ICWSM.Google ScholarCross Ref
- Donghee Yvette Wohn. 2019. Volunteer Moderators in Twitch Micro Communities : How They Get Involved, the Roles They Play, and the Emotional Labor They Experience. In Proceedings of CHI.Google ScholarDigital Library
- Austin P. Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Diyi Yang, and Duen Horng Chau. 2021. RECAST : Interactive Auditing of Automatic Toxicity Detection Models. In Proceedings of CSCW.Google Scholar
- Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex Machina : Personal Attacks Seen at Scale. In Proceedings of WWW.Google ScholarDigital Library
- Justine Zhang, Jonathan P. Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Nithum Thain, Yiqing Hua, and Dario Taraborelli. 2018. Conversations Gone Awry : Detecting Early Signs of Conversational Failure. In Proceedings of ACL.Google ScholarCross Ref
- Justine Zhang, Arthur Spirling, and Cristian Danescu-Niculescu-Mizil. 2017. Asking Too Much ? The Rhetorical Role of Questions in Political Discourse. In Proceedings of EMNLP. ioGoogle ScholarCross Ref
Index Terms
- Thread With Caution: Proactively Helping Users Assess and Deescalate Tension in Their Online Discussions
Recommendations
Proactive Moderation of Online Discussions: Existing Practices and the Potential for Algorithmic Support
CSCWTo address the widespread problem of uncivil behavior, many online discussion platforms employ human moderators to take action against objectionable content, such as removing it or placing sanctions on its authors. Thisreactive paradigm of taking action ...
Third-person perception of online comments
Using online comments posted on news stories as the context, this study aimed to examine the interplay between the third-person perception (TPP) - that people believe media message have a greater effect on changing the attitudes of others compared with ...
Students' Sense of Belonging in Online Classes: Does Age Matter?
This study explored graduate students' perceptions of their sense of belonging in online classes, their comfort in participating in online discussions, and their technology skills. Differences in these areas among three different age groups were ...
Comments