The president’s post inspired a lot of debate about what counts as an incitement to violence, about whether you could kick a head of state off your platform, and about whether platforms are getting played by fascists. Those are all good discussions to have, though they mostly seem to avoid a larger point, which is that an enormous amount of political speech now transpires on a platform used by 1.73 billion people a day, and whose rules about speech are ultimately decided and enforced by a single person: Facebook’s CEO, Mark Zuckerberg, who controls the majority of the company’s stock.
Later this year you might have some recourse
if Facebook makes a speech-related decision that doesn’t go your way. But you don’t have any recourse now, and this is one reason why when people get mad at Facebook, they tend to get really
mad at Facebook. You know what they say about the relationship between justice and peace
: you can’t have one without the other, and while Facebook’s content moderation is good at a lot of things it feels like a stretch to call it just
One thing you could do if you wanted to distribute power over speech more broadly is to entrust some of it to your users. A company that has done this with some success is Reddit, which sets a “floor” of speech rules
(no spam!) but allows individual forums (called subreddits) to raise the “ceiling.”
A religious subreddit might ban cursing, for example. Or a forum related to thoughts had in the shower might ban the publication of non-shower thoughts
. Speech is much more easily moderated when people in the room have shared context around values and interests, and so “I want to read what thoughts people have in the shower” is a discussion more effectively policed than “let’s see what 1.73 billion people have to say about current affairs.”
As protests over the killing of George Floyd by police have spread around the world, lots of companies have made statements affirming a new commitment to the black community and its allies. Reddit issued such a statement last week
. The company said it would honor the request of departing co-founder Alexis Ohanian
to replace him on the board with a black person. And it acknowledged that historically it has been a haven for the spread of racist ideology. Here’s CEO Steve Huffman:
While we dealt with many communities themselves, we still did not provide the clarity — and it showed, both in our enforcement and in confusion about where we stand. In 2018, I confusingly said racism is not against the rules, but also isn’t welcome on Reddit
. This gap between our content policy and our values has eroded our effectiveness in combating hate and racism on Reddit; I accept full responsibility for this. […]
Despite making significant progress over the years, we have to turn a mirror on ourselves and be willing to do the hard work of making sure we are living up to our values in our product and policies. This is a significant moment. We have a choice: return to the status quo or use this opportunity for change. We at Reddit are opting for the latter, and we will do our very best to be a part of the progress.
The furor has calmed since then, but most of the hard questions remain unanswered. How do you attract, train and retain an army of volunteers to moderate racist speech across every forum on Reddit? How much of the responsibility should belong to individual moderators, and how much should belong to Reddit itself? How much of Reddit’s delegation of authority was really just a dereliction of duty?
“So much of what is happening now lies at your feet,” Pao told Huffman
. “You don’t get to say BLM when Reddit nurtures and monetizes white supremacy and hate all day long.”
Of course, decentralizing moderation the way Reddit does isn’t a new idea to Facebook. In fact, the company employs a similar approach in Facebook Groups, the company’s own take on smaller forums. Zuckerberg has said
that groups represent a top priority for the company, and so it feels like a good time to check in on the state of moderation there.
As on Reddit, Facebook group moderators can raise the ceiling for moderation to shape the discussion and ban certain kinds of posts. In The Verge
, Ashley Carman has an important story about how Facebook groups have handled the recent surge of content related to Black Lives Matter
. More conservative moderators have taken a heavy-handed approach to removing such posts — no non-shower thoughts in the shower thoughts forum
! But that has outraged users who see the ongoing protests and the injustices they are drawing attention to as matters of basic human rights, worthy of discussion in whatever context the speaker finds relevant.
Boss-Moms is one of many Facebook groups grappling with inadequate moderation policies as members attempt to discuss Black Lives Matter. The groups, which range in focus from video games to music to local communities, are moderated by other group members. The moderators have no formal training from Facebook or outside sources and make their own decisions about what content is and is not allowed. Most groups have no reference point for how to give everyone a voice, and that’s led to fighting between members, people leaving, groups temporarily shutting down, and splinter groups breaking off.
Many groups don’t have people of color as moderators, adding to the moderation problem. Roop Mangat belongs to a pair of local community Facebook groups — one in which no people of color or women are moderators, and another that has a white woman moderator who serves alongside only white men. Mangat posted the same message to both groups, urging people to take racism in the community seriously. One of them deleted it. “They think anything ‘political’ is not appropriate,” she tells me over Twitter DM. “Yet they still allow posts that include gossip and false information to spread.” The group threatened to ban her if she posted again.
A similar dynamic is playing out on the neighborhood social network NextDoor, where a let-the-users-do-it approach to moderation has left many black users feeling fearing for their safety. Makena Kelly talked to some of those folks at The Verge
, and uncovered how the company’s all-too-convenient embrace of community moderation has left it with significant blindspots.
This hands-off approach is what makes Nextdoor able to be as big as it is. By outsourcing moderation to untrained and unpaid volunteers, the company has been able to expand into over 200,000 neighborhoods
across the country. But it has also empowered community members to strike down posts they personally don’t like. All across the country, Nextdoor posts advertising protests get struck down by community moderators while racist and inflammatory messages, some calling for direct violence against black people and protestors, are left to stand
Leads don’t go through any formal training from Nextdoor before receiving the authority to strike posts, and the guidelines listed on the site are vague enough for leads to interpret them in different ways. There are no rules promoting diversity in moderation leadership either. In a private forum — known as the National Leads Forum, as first reported by BuzzFeed News
— some community moderators were enraged by Nextdoor’s decision to support the Black Lives Matter movement. Around the same time the company issued its public statement last week, that same language was published on Nextdoor feeds, enraging some moderators active in the private forum.
“I would like to see Nextdoor post a ‘White lives matter’ [post],’” one moderator from Orlando, Florida, wrote. “Sometimes, we need to remember ‘All lives matter!’”
To recap: one approach to content moderation is to entrust all of human speech to a single person, putting the user base at the mercy of their decisions without any practical recourse. Another is to entrust moderation to a group of unpaid volunteers, without much regard to that group’s diversity, or training, or values, and hope that they don’t embarrass you too much along the way.
Maybe you can help improve the former system by setting up an independent body of experts to offer binding opinions on controversial subjects. And maybe you can improve the latter by offering volunteers more training, more mental health resources, and actual money for their labor.
But as discussions about racism and injustice light up forums around the internet, it’s clear that the mechanisms companies have built to date are often failing to support that conversation. As a result, offline injustices are being replicated in online speech. As Silicon Valley reconsiders how companies can support black people, their allies, and other minority communities, moderation ought to be front and center.