In the 2018 documentary “The Cleaners,” a young man in Manila, Philippines, explains his work as a content moderator: “We see the pictures on the screen. You then go through the pictures and delete those that don’t meet the guidelines. The daily quota of pictures is 25,000.” As he speaks, his mouse clicks, deleting offending images while allowing others to remain online.
The man in Manila is one of thousands of content moderators hired as contractors by social media platforms – 10,000 at Google alone. Content moderation on an industrial scale like this is part of the everyday experience for users of social media. Occasionally a post someone makes is removed, or a post someone thinks is offensive is allowed to go viral.
Similarly, platforms add and remove features without input from the people who are most affected by those decisions. Whether you are outraged or unperturbed, most people don’t think much about the history of a system in which people in conference rooms in Silicon Valley and Manila determine your experiences online.
But why should a few companies – or a few billionaire owners – have the power to decide everything about online spaces that billions of people use? This unaccountable model of governance has led stakeholders of all stripes to criticize platforms’ decisions as arbitrary, corrupt or irresponsible. In the early, pre-web days of the social internet, decisions about the spaces people gathered in online were often made by members of the community. Our examination of the early history of online governance suggests that social media platforms could return – at least in part – to models of community governance in order to address their crisis of legitimacy.
The documentary ‘The Cleaners’ shows some of the hidden costs of Big Tech’s customer service approach to content moderation.
Online governance – a history
In many early online spaces, governance was handled by community members, not by professionals. One early online space, LambdaMOO, invited users to build their own governance system, which devolved power from the hands of those who technically controlled the space – administrators known as “wizards” – to members of the community. This was accomplished via a formal petitioning process and a set of appointed mediators who resolved conflicts between users.
Other spaces had more informal processes for incorporating community input. For example, on bulletin board systems, users voted with their wallets, removing critical financial support if they disagreed with the decisions made by the system’s administrators. Other spaces, like text-based Usenet newsgroups, gave users substantial power to shape their experiences. The newsgroups left obvious spam in place, but gave users tools to block it if they chose to. Usenet’s administrators argued that it was fairer to allow each user to make decisions that…
