Publications

Our  lab focuses on producing novel research in the area of A.I. based civic technology. Within this space, we are primarily focused on studying and designing technology that impacts gig work and disinformation.

In the area of gig work we focus on: (a) understanding the labor conditions of workers on gig platforms to identify the main challenges and problems that workers face. Given that currently gig labor platforms are blackboxes to researchers we conduct interviews, surveys, and also develop novel tools that allow us to understand workers’ labor conditions; (b) design novel tools to empower workers and establish better work collaborations and engagements with the other involved stakeholders (employers, platform owners, governments, NGOs, academics). 

In the area of disinformation we study large scale political disinformation data from the US and Latin America. We then design tools, systems, and interventions that can help to combat political disinformation at the national level with governments, NGOs, and industry.

Jump to: 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2010

Research Papers, Policy Briefs, and Books

2021

Quantifying the Invisible Labor in Crowd Work
Carlos Toxtli, Siddharth Suri, Saiph Savage
CSCW: ACM Conference on Computer-Supported Cooperative Work 2021.
[PDF]

A National Cloud for Conducting Disinformation Research at Scale
Saiph Savage, Cristina Martinez
Policy Brief for the Federation of American Scientists 2021.
[PDF]

Fighting Disaster Misinformation in Latin America:The #19S Mexican Earthquake Case Study
Claudia Flores-Saviaga, Saiph Savage
Springer Journal on Personal and Ubiquitous Computing 2021
[PDF]

The Expertise Involved in Deciding which HITs are Worth Doing on Amazon Mechanical Turk
Benjamin Hanrahan, Anita Chen, Jiahua Ma, Ning Ma, Anna Squicciarini, Saiph Savage
CSCW: ACM Conference on Computer-Supported Cooperative Work 2021.
[PDF]

Research Methods to Study & Empower Crowd Workers
Saiph Savage, Carlos Toxtli, Eber Betanzos
Book Chapter in Research Methods for Digital Work and Organization,
Oxford University Press 2021.
[PDF]

How to Apply Artificial Intelligence to Streamline Government Procedures?
Saiph Savage, Norma Elva Chávez, Saúl Esparza, Diego Flores,
Shazia Ansari, Alexandre Turcotte, Homero Piedra
White Paper for the Inter-American Development Bank (IDB) 2021.
[PDF]

Flexible Work and Personal Digital Infrastructures
Mohammad H Jarrahi, Newlands, G., Butler, B., Saiph Savage,
Lutz, C., Dunn, M., Sawyer Steven.
CACM: Communications of the ACM 2021.
[PDF],[VIDEO] 

A Transformation with Artificial Intelligence
Africa Sahara, Saiph Savage
Audio Book, Published by Beek 2021.
[AUDIO BOOK]

Data Labeling Work in the AI Ecosystem and Opportunities for Improvement
Matthew Lease, Mohammad Hossein Jarrahi, and Saiph Savage
Rabb Symposium on Embedding AI in Society 2021.
[PDF]

Goodbye, Parler! A story about digital platforms, restrictions, and Alt-Tech
Animal Politico Essay 2021.
[PDF]

 

2020

Becoming the Super Turker: Increasing Wages via a Strategy from High Earning Workers
Saiph Savage, Chun-Wei Chiang, Susumu Saito, Carlos Toxtli, Jeffrey Bigham
WWW: The Web Conference 2020
[PDF],[HTML],[SLIDES], [DATASET AND TOOLS]

Cyberbullying in Mexico
Saiph Savage, Norma Elva Chavez, Gabriela Huerta
Book Chapter in Examining Internet and Technology around the World, Global Viewpoints 2020.
[PDF]

Reputation Agent: Prompting Fair Reviews in Gig Markets
Carlos Toxtli, Angela Richmond-Fuller, Saiph Savage
WWW: The Web Conference 2020
[PDF],[HTML],[SLIDES], [DATASET AND TOOLS]

The Challenges of Crowd Workers in Rural and Urban America
Claudia Flores-Saviaga, Yuwen Li, Benjamin Hanrahan, Jeffrey Bigham, Saiph Savage
HCOMP: AAAI Conference on Human Computation and Crowdsourcing 2020
[PDF],[TALK]

Solidarity and A.I. for Transitioning to Crowd Work during COVID-19
Saiph Savage, Mohammad H Jarrahi
The New Future of Work Symposium 2020
[PDF],[VIDEO]

Citizens as More Than Sensors, Citizens as Agents for Change
Saiph Savage
WWW: The Web Conference 2020
[PDF], [VIDEO TALK]

Enabling Expert Critique at Scale with Chatbots and Micro-Guidance
Carlos Toxtli, Saiph Savage
ACHI: Advances in Computer-Human Interactions 2020
[PDF]

Reciprocal Research: Providing Value in Design Research from the Outset in the Rural United States
Benjamin Hanrahan, Ning Ma, Eber Betanzos, Saiph Savage
ICTD:Conference on Information and Communication Technologies and Development 2020
[PDF]

Predicting the Working Time of Microtasks Based on Workers’ Perception of Prediction Errors
Susumu Saito, Chun-Wei Chiang, Saiph Savage, Jeffrey Bigham
Journal of Human Computation 2020
[PDF]

Meta-Gig: Empowering Anyone to Create Crowd Marketplaces
Carlos Toxtli, Saiph Savage
Avances en Interacción Humano Computadora 2020.
[PDF]

Understanding Knowledge Production for NGOs with Paid Crowd Workers and Online Volunteers
Claudia Flores-Saviaga, Ricardo Granados, Liliana Savage,
Lizbeth Escobedo, Saiph Savage
Avances en Interacción Humano Computadora 2020.
[PDF]

Spanish-Language COVID Fact-Checking Spun for Political Purposes
Saiph Savage, Claudia Flores-Saviaga, Rafael Morales, Esteban Ponce de León,
White Paper for the Atlantic Council 2020
[PDF], [HTML]

Understanding the Threats and Consequences of Computational Propaganda on Native Americans
Amy Ruckes, Norma Elva Chavez, Eber Betanzos, Saiph Savage,
White Paper for Mexico’s Electoral Institute 2020
[PDF]

Using Smart Contracts for Governance and Identity
Alberto Garcia, Cameron Dennis, Norma Elva Chavez, Amy Ruckes,
Eber Betanzos, Saiph Savage
FAB’20: Symposium on Foundations and Applications of Blockchain 2020
[PDF],[Video]

Integrating Culture Theory and HCD to Design Global Government Interfaces
Anqui Cao, Antonio Aranda-Eggermont, Mojin Yu, Patriya Wiesmann, Stephanie Blucker, Shazia Ansari, Diego Flores, Fabian Medina, Saiph Savage
CSCW Workshop Paper in the Civic Technologies Workshop 2020.
[PDF]

Diseñando Asistentes Virtuales Inteligentes para el Gobierno Federal
Saul Esparza, Norma Elva Chavez, Diego Flores, Fabian Mediana, Saiph Savage
CSCW Workshop Paper in the Civic Technologies Workshop 2020.
[PDF]

Exploitation of Native American Social Media for Political Propaganda
Amy Ruckes, Eber Betanzos, Saiph Savage,
41st American Indian Workshop 2020
[PDF]

Virtual Assistant for Latinas Experiencing Domestic Violence
Liliana Savage, Luis Cervantes, Xue Chen, Ricardo Granados, Saiph Savage
CSCW Workshop Paper in the Civic Technologies Workshop 2020.
[PDF]

 

2019

Turker Tales: Integrating Tangential Play into Crowd Work
Anna Kasunik, Chun-Wei Chiang, Geoff Kaufman, Saiph Savage
DIS: ACM Conference on Designing Interactive Systems 2019
[PDF]

Audience and Streamer Participation at Scale on Twitch
Claudia Flores Saviaga, Jessica Hammer, Juan Pablo Flores, Saiph Savage
HT: ACM Conference on Hypertext and Social Media 2019
[PDF]

TurkScanner: Predicting the Hourly Wage of Microtasks
Susumu Saito, Chun-Wei Chiang, Saiph Savage, Jeffrey Bigham
WWW: The Web Conference 2019
Honorable Mention 🏆
[PDF]

Deploying Real World Blockchain Systems
Chun Wei Chiang, Eber Betanzos, Saiph Savage
Journal of Cyberspace Studies 2019.
[PDF]

BOTS: For Building a Better Society?
Liliana Savage, Eber Betanzos, Saiph Savage
Book Chapter for CIDOB International Yearbook 2019.
[PDF]

Understanding the Crowd Markets that Workers and Requesters Imagine
Carlos Toxtli, Juan Pablo Castillo, Alberto Campos, et al. Saiph Savage
CI: ACM Conference on Collective Intelligence 2019
[PDF]

Anti-LatinX: Computational Propaganda in the US
Claudia Flores-Saviaga, Saiph Savage
Institute of the Future White Paper 2019.
[PDF] [BuzzFeed News]

Native Americans and the Threat of Computational Propaganda
Amy Ruckes, Eber Betanzos, Saiph Savage
White Paper for the Atlantic Council 2019.
[PDF]

Unraveling the Operations in Latin America of the Russian State-Controlled International Television Network (RT).
Liliana Savage, Claudia Flores-Saviaga, Saiph Savage
White Paper for the London School of Economics 2019.
[PDF]

Org-scale analytics: Today’s startups build societies. Do it right.
Seth Frey, Grace Benefield, Clark Bernier, Maarten Bos, Ceren Budak, Simon DeDeo, Rosta Farzan, Benjamin Mako Hill, Abigail Jacobs, Saiph Savage, Aaron Shaw
Towards Data Science Journal 2019
[PDF]

Blockchain for Governance and Civic Participation in Mexico.
Liliana Savage, Eber Betanzos, Saiph Savage
White Paper 2019.
[PDF]

Worker Demographics and Earnings on Amazon Mechanical Turk: An Exploratory Analysis
Kotaro Hara, Kristy Milland, Benjamin Hanrahan,
Chris Callison-Burch, Abigail Adams, Saiph Savage, Jeffrey Bigham.
CHI: ACM Conference on Human Factors in Computing Systems 2019
[PDF] [BLOG POST] [TALK] [BIBTEX]

 

2018

Crowd Coach: Peer Coaching for Crowd Workers’ Skill Growth
Chun-Wei Chiang, Anna Kasunic, Saiph Savage
CSCW: ACM Conference on Computer-Supported Cooperative Work 2018.
[PDF] [BLOG POST] [TALK] [BIBTEX]

The Social Roles of Bots: Evaluating Impact of Bots on Discussions in Online Communities
Joseph Seering, Juan Pablo Flores, Saiph Savage, Jessica Hammer
CSCW: ACM Conference on Computer-Supported Cooperative Work 2018.
[PDF] [BLOG POST] [TALK] [BIBTEX]

Mobilizing the Trump Train: Understanding Collective Action in a Political Troll Community
Claudia Flores-Saviaga, Brian Keegan, Saiph Savage
ICWSM: International AAAI Conference on Web and Social Media 2018
[PDF] [BLOG POST] [TALK] [BIBTEX]

A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk
Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, Jeffrey Bigham
CHI: ACM Conference on Human Factors in Computing Systems 2018
Honorable Mention (Top 5% of all CHI Papers) 🏆
[PDF] [BLOG POST] [TALK] [BIBTEX]

Crowdsourcing the Installation and Maintenance of Indoor Localization Infrastructure to Support Blind Navigation
Cole Gleason, Dragan Ahmetovic, Saiph Savage, Carlos Toxtli, Carl Posthuma, Chieko Asakawa, Kris Kitani, Jeffrey Bigham
UbiComp Journal: ACM Journal on Interactive, Mobile, Wearable and Ubiquitous Technologies 2018
[PDF]

Blockchain for Trustful Collaborations between Immigrants and Governments
Chun-Wei Chiang, Eber Betanzos, Saiph Savage
CHI: ACM Conference on Human Factors in Computing Systems 2018
[PDF] [NEWS COVERAGE] [TALK] [BIBTEX]

Enabling Expert Critique with Chatbots and Micro Guidance
Carlos Toxtli, Joel Chan, Walter S. Lasecki, Saiph Savage
CI: ACM Collective Intelligence Conference 2018
[PDF] [BLOG POST] [TALK] [BIBTEX]

Crowd Work on a CV? Understanding How AMT Fits into Turkers’ Career Goals and Professional Profiles
Anna Kasunic, Chun Wei Chiang, Geoff Kaufman, Saiph Savage
CI: ACM Collective Intelligence Conference 2018
[PDF] [BLOG POST] [TALK] [BIBTEX]

User Experiences with Traditional and 3D-Printed Upper Extremity Prostheses, Development of a Comprehensive Survey Instrument
Goeran Fiedler, Jennifer Mankoff, Sydney Eckert, Chelsea Ngo, Saiph Savage
Canadian Prosthetics and Orthotics Journal 2018
[PDF] [BLOG POST] [TALK] [BIBTEX]

Designing Blockchain Technology to Transform Rural Communities
Chun-Wei Chiang, Eber Betanzos, Michael Austin, Saiph Savage
Workshop CHI: ACM Conference on Human Factors in Computing Systems 2018
[PDF] [TALK] [BIBTEX]

Detecting and Counter-Attacking Online Radicalization and Cyberwars
Saiph Savage, Eber Betanzos
POST 2018 (DOD Funding conference)
[PDF]

 

2017

Subcontracting Microwork
Meredith Ringel Morris, Jeffrey P. Bigham, Robin Brewer, Jonathan Bragg, Anand Kulkarni, Jessie Li, and Saiph Savage
CHI: ACM Conference on Human Factors in Computing Systems 2017
[PDF] [BLOG POST] [TALK] [BIBTEX]

The Effects of “Not Knowing What You Don’t Know” on Web Accessibility for Blind Web Users
Jeffrey Bigham, Irene Lin, Saiph Savage
ASSETS: ACM Conference on Computers and Accessibility 2017
[PDF] [TALK] [BIBTEX]

Audience Participation Games: Blurring the Line Between Player and Spectator
Joseph Seering, Saiph Savage, Michael Eagle, Joshua Churchin, Rachel Moeller, Jeffrey Bigham, Jessica Hammer
DIS: ACM Conference on Designing Interactive Systems 2017
[PDF] [BLOG POST] [TALK] [BIBTEX]

Understanding Mobile Money Perceptions of Latin American Migrants
Chun-Wei Chiang, Claudia Flores Saviaga, Eduardo Jr Arenas, Felipe Colin, Mario Romero, Cuauhtemoc Rivera-Loaiza, Norma Elva Chavez and Saiph Savage
Latin American Conference on Human-Computer Interaction 2017
[PDF] [TALK] [BIBTEX]

On How Deaf People Might Use Speech to Control Devices
Jeffrey Bigham, Raja Kushalnagar, Ting-Hao K. Huang, Karen Kim, Juan Pablo Flores, Saiph Savage
ASSETS: ACM Conference on Computers and Accessibility 2017
[PDF] [BIBTEX]

Automated Assistants to Identify and Prompt Action on Visual News Bias
Vishwajeet Narwa, Mohamed Hashim Salih, Jose Angel Lopez, Angel Ortega, John o’Donovan, Tobias Hollerer, and Saiph Savage
CHI: ACM Conference on Human Factors in Computing Systems 2017
[PDF] [BIBTEX]

CHI-nnabis: Implications of Marijuana Legalization for and from Human-Computer Interaction
Brian C. Keegan, Jofish Kaye, Patricia Cavazos-Rehg, Munmun de Choudhury, Dong Nguyen, Michael J. Paul, and Saiph Savage
CHI: ACM Conference on Human Factors in Computing Systems 2017
[PDF]

 

2016

Social Media, Civic Engagement, and the Slactivism Hypothesis: Lessons from Mexico’s “El Bronco”
Phil Howard, Saiph Savage, Claudia Saviaga, Carlos Toxtli, Andres Monroy-Hemández
Journal Of International Affairs 2016
[PDF] [MEDIA] [BIBTEX]

Botivist: Calling Volunteers to Action Using Online Bots
Saiph Savage, Andres Monroy-Hernandez, Tobias Hollerer
CSCW: ACM Conference on Computer-Supported Cooperative Work 2016
[PDF] [MEDIA] [BIBTEX] [TALK]

Bring on Board New Enthusiasts! A Case Study of Impact of Wikipedia Art + Feminism Edit-a-thon Events on Newcomers
Rosta Farzan, Saiph Savage, Claudia Flores-Saviaga
SocInfo: 8th International Conference on Social Informatics 2016
[PDF] [MEDIA] [BIBTEX] [TALK]

Innovations in Autonomous Systems: Challenges and Opportunities for Human-Agent Collaboration
Lia Emanuel, Joel Fischer, Wendy Ju, Saiph Savage
CSCW: ACM Conference on Computer-Supported Cooperative Work 2016
[PDF] [MEDIA] [BIBTEX] [TALK]

How To Think About Bots
Sam Woolley, danah boyd, Gilad Lotan, Gilad Rasner, Samantha Shorey, Alexis Lloyd, Luis Daniel Palacios, Saiph Savage, Meredith Broussard, Madeline Clare Elish, Lainna Fader, Tim Hwang
Vice Magazine
[PDF]

BotViz: Data Visualizations for Collaborations With Bots and Volunteers
Carlos Toxtli, Claudia Flores-Saviaga, Flor Aguilar, Alejandra Monroy, William Dai, Juan Pablo Flores, Nedda Amini, Jeerel Herrejon, Shloka Desai, Norma Elva Chavez, Saiph Savage
CSCW: ACM Conference on Computer-Supported Cooperative Work 2016
[PDF] [MEDIA] [BIBTEX] [TALK]

Tag Me Maybe: Perceptions of Public Targeted Sharing on Facebook
Saiph Savage, Andres Monroy-Hernandez,Kasturi Bhattacharjee, Tobias Hollerer.
Hypertext:26th ACM Conference on Hypertext and Social Media 2015
[PDF] [MEDIA] [BIBTEX] [TALK]

LeadWise: Using Online Bots to Recruite and Guide Expert Volunteers
Claudia Flores-Saviaga, Saiph Savage, Dario Taraborelli
CSCW: ACM Conference on Computer-Supported Cooperative Work 2016
[PDF] [MEDIA] [BIBTEX] [TALK]

 

2015

Participatory Militias: An Analysis of an Armed Movement’s Online Audience.
Saiph Savage, Andres Monroy-Hernandez.
CSCW: ACM Conference on Computer-Supported Cooperative Work 2015
[PDF] [MEDIA] [BIBTEX] [TALK]

Political Bots and the Manipulation of Public Opinion in Venezuela
Michelle Forelle, Phil Howard, Andres Monroy-Hernandez, Saiph Savage.
arXiv 2015
[PDF] [MEDIA] [BIBTEX] [TALK]

Daemo: a Self-Governed Crowdsourcing Marketplace
Stanford Research Collective, Saiph Savage, Michael Bernstein.
UIST: ACM Conference on User Interface Software and Technology 2015
[PDF] [MEDIA] [BIBTEX]

Motivating Crowds to Volunteer Neighborhood Data
Nataly Moreno, Saiph Savage, Anamary Leal, Jessica Cornick, Matthew Turk, Tobias Hollerer.
CSCW: ACM Conference on Computer-Supported Cooperative Work 2015
[PDF] [BIBTEX]

Participatory Stoves: Designing Renewable Energy Technologies for the Rural Sector
Walter Angel, Saiph Savage, Nataly Moreno.
CSCW: ACM Conference on Computer-Supported Cooperative Work 2015
[PDF] [PHOTOS] [BIBTEX] [MEDIA]

 

2014

Visualizing Targeted Online Audiences
Saiph Savage, Angus Forbes, Carlos Toxtli, Grant McKenzie, Shloka Desai, Tobias Hollerer.
COOP: Conference on the Design of Cooperative Systems 2014: Springer
[PDF] [VIDEO] [BIBTEX] [TALK]

The “Courage For” Facebook Pages: Advocacy Citizen Journalism in the Wild.
Saiph Savage, Andres Monroy-Hernandez.
C+J:Computation+Journalism Symposium 2014. American Journalism Review.
[PDF] [MEDIA] [BIBTEX]

FiberKraft: An Interface for Designing Optical Fiber Networks
Julio Cruz, Hugo Estrada, Saiph Savage, Norma Elva Chavez.
ENC: Encuentro Nacional de Ciencias de la Computacion 2014}
[PDF] [VIDEO] [BIBTEX] [MEDIA]

I’m Feeling LoCo: A Location Based Context Aware Recommendation System.
Norma Saiph Savage, Maciej Baranski, Norma Elva Chavez, Tobias Hollerer.
LBS: International Symposium on Location-Based Services. Lecture Notes in Geoinformation and Cartography. Springer
[PDF] [BIBTEX] [TALK]

Crowdsourcing Volunteers
Shloka Desai, Luis Rodriguez, Saiph Savage, Norma Elva Chavez, Angus Forbes.
SoCal CWIC: Celebration of Women in Computing in Southern California 2014.
[PDF]

Socially and Contextually Appropriate Recommendation Systems
Saiph Savage, Lama Nachman, Saurav Sahay, Giuseppe Raffa.
U.S. Patent Disclosure, Filed 2014
[PDF]

 

2013

Online Social Persona Management
Saiph Savage, Rita Wouhaybi, Lama Nachman, Saurav Sahay.
U.S. Patent Disclosure, Filed 2013
[PDF]

Traversing Data Using Data Relationships
Saiph Savage, Rita Wouhaybi.
U.S. Patent Disclosure, Filed 2013
[PDF]

Utilizing Crowdsourced Databases for Social Media Question Asking
Saiph Savage, Angus Forbes, Rodrigo Savage, Tobias Hollerer, Norma Elva Chavez.
CSCW: Workshop on Social Media Question Asking at ACM Conference on Computer-Supported Cooperative Work 2013
[PDF]

Directed Social Queries with Transparent User Models
Saiph Savage, Angus Forbes, Rodrigo Savage, Tobias Hollerer, Norma Elva Chavez.
UIST: ACM Symposium on User Interface Software and Technology 2012
[PDF] [VIDEO]

Visualizing and Verifying Directed Social Queries
Angus Forbes, Saiph Savage, Tobias Hollerer.
InfoVis: IEEE Workshop on Interactive Visual Text Analytics at IEEE Transactions on Visualization and Computer Graphics 2012
[PDF] [VIDEO]

Search on the Cloud File System
Rodrigo Savage,Tania Nava, Norma Elva Chavez, Norma Saiph Savage.
PDCS: Parallel and Distributed Computing and Systems Conference 2011
[PDF]

Nintendo Wiimote for Arm and Wrist Therapy in Stroke Survivors with Upper Extremity Hemipariesis
Ron S. Leder, Gil Azcarate, Rodrigo Savage, Saiph Savage, L. Enrique Sucar, David Reinkensmeyer, Carlos Toxtli, Emilio Roth, Ariel Molina.
IEEE Virtual Rehabilitation 2008
[PDF] [BIBTEX]

A Social Crowd-Controlled Orchestra
Saiph Savage, Norma Elva Chavez, Carlos Toxtli, Salvador Medina, David Alvarez, Tobias Hollerer.
CSCW: ACM Conference on Computer-Supported Cooperative Work 2013
[PDF]

 

2012

Enchantment Under the Sea: An Intelligent Environment for User Friendly Music Mixing
Norma Elva Chavez, Rodrigo Savage, Tania Nava, Saiph Savage.
IE: International Conference on Intelligent Environments 2012
[PDF]

Never Alone:A Crowdbased Interactive Musical Interface.
Norma Elva Chavez, Rodrigo Savage, Saiph Savage.
SoCal CWIC: Celebration of Women in Computing in Southern California 2012
[PDF]

 

2010

Mmmmm:A multi-modal mobile music mixer
Norma Saiph Savage, Reza Ali, Norma Elva Chavez, Rodrigo Savage.
NIME: Conference on New Interfaces for Musical Expression 2010
[PDF] [BIBTEX]

Frequent trajectory mining on GPS data.
Norma Saiph Savage, Shoji Nishimura, Norma Elva Chavez, and Xifeng Yan.
LocWeb: ACM Workshop on Location and the Web, at the Internet of Things, 2010.

Right place at the right time: novel place recommendation system
Norma Elva Chávez, Saiph Savage, Rodrigo Savage.
IX Congreso Internacional sobre Innovación y Desarrollo Tecnológico, Cuernavaca, México.