Facebook has faced ongoing criticism and scrutiny over its content moderation practices. With billions of users posting content daily, reviewing and enforcing content policies is an immense challenge. Recent reports have claimed that Facebook’s moderation systems make over 300,000 mistakes per day. But what does this really mean?
The Claim: 300,000 Mistakes Per Day
In May 2021, The New York Times published an article detailing struggles within Facebook’s content moderation teams. The article stated:
The company estimates that it is removing more than three million pieces of content a day across its platforms. But among the terabytes of posts containing everything from child sexual abuse to terrorism, Facebook’s automated systems and human moderators err up to 300,000 times a day.
This claim of 300,000 errors per day was widely reported and cited as evidence of major flaws in Facebook’s moderation. However, there are several important caveats.
Firstly, the number comes from Facebook’s own internal estimate referenced in a leaked memo. It has not been externally verified. Secondly, what counts as a “mistake” is not clearly defined. The statistic likely includes any action taken on content that is later overturned on appeal, ranging from minor errors to complete misjudgements.
Understanding Facebook’s Scale
To put the 300,000 figure in context, it’s vital to comprehend the enormous scale of content posted to Facebook daily:
- Over 3 billion people use Facebook monthly
- 500+ million Facebook stories are shared every day
- 300+ million photos are uploaded every day
- 5 billion pieces of content are shared on WhatsApp daily
With this huge firehose of content, even a 99.99% success rate would still yield thousands of daily errors.
The Moderation Process
Facebook uses a blended approach to moderating content:
- Automated systems – AI and machine learning instantly review posts and remove prohibited content at scale.
- User reports – Users can report offensive or rule-breaking posts for review.
- Human moderators – Teams review complex context-based cases escalated from automated systems or user reports.
Mistakes can happen at each stage, from AI false positives to poor judgment calls by people. The volume handled by automatic systems means tiny failure rates still add up.
Key Factors in Content Moderation
Below are some of the inherent challenges involved in moderating content at Facebook’s size:
Nuanced Rules
Facebook’s content policies prohibit broad categories like hate speech, harassment, and misinformation. But making consistent judgments requires interpreting complex context and nuance human moderators can struggle with.
Language and Cultural Knowledge
With users worldwide, moderators must understand local cultural norms and speak languages of the content they review. Without this context, errors are inevitable.
Evolving Nature of Policies
Content policies continuously adapt to address new issues. But retroactively applying updated rules leads to previously acceptable content now being considered policy violations.
Subjectivity of Human Review
No two people will interpret the exact same piece of content identically. Disagreements between moderators and appeals teams will always generate some proportion of “mistakes”.
Incentives for Gaming the System
When people want prohibited content to remain online, they have incentives to overwhelm moderators and exploit loopholes. This pushes error rates higher despite best efforts.
Potential Impacts of Inaccurate Moderation
Both false positives and false negatives in content moderation can have real consequences:
Legitimate Posts Being Removed
Mistakenly categorizing valid content as rule-breaking suppresses voices and impedes free expression.
Harmful Content Remaining Online
Failing to remove toxic, dangerous, or illegal material enables its proliferation and damages vulnerable groups.
Eroded Trust in Platforms
Highly visible moderation mistakes contribute to users’ perception of systemic bias, incompetence, or dishonesty.
Weaponization of Reporting Systems
If reporting systems are viewed as unreliable, they get exploited to arbitrarily silence or harm other users.
Reinforcing Bias
If certain demographics are more frequently falsely accused, marginalized groups experience disproportionate censorship.
Changes Being Made by Facebook
Facebook claims it is continuously improving its approach:
- Increasing content reviewer staffing to over 35,000 people
- Advancing AI capabilities to handle more nuanced evaluations
- Updating policies based on expert/stakeholder consultations
- Enhancing content evaluation training for reviewers
- Expanding appeals options for users
Despite these efforts, with billions of users the challenges around mistakes at scale remain daunting.
Evaluating the 300,000 Mistakes per Day Claim
Given the available information, how concerned should we be about the reported 300,000 daily moderation mistakes?
To help put the figure in perspective, the table below compares Facebook’s daily user activity statistics against the alleged error rate:
| Facebook Statistic | Daily Quantity |
|---|---|
| Monthly active users | 3 billion |
| Daily active users | 1.9 billion |
| Stories shared daily | Over 500 million |
| Photos uploaded daily | Over 300 million |
| Estimated moderation mistakes | 300,000 |
Given Facebook’s scale, even with a robust moderation system, some fractional error rate is likely unavoidable. A mistake incidence of 0.01% relative to photos shared and 0.06% of stories shared, while still significant, is arguably less alarming in context.
However, the true harm in incorrect moderation depends on mistake severity, not just quantity. Even a single instance of removing valid activism content or permitting dangerous misinformation can have serious repercussions.
Key Uncertainties
There are also notable gaps in our understanding:
- The methodology behind Facebook’s 300,000 self-reported mistake estimate is unclear.
- The severity of errors and subsequent impacts is unknown.
- Whether the rate is stable or changing over time as processes improve has not been shown.
- Comparison with other platforms’ performance benchmarks is not available.
Conclusion
On balance, the claim of 300,000 daily moderation mistakes by Facebook seems plausible given the enormous volume of content. However, without greater transparency into the nature and repercussions of these errors, it is hard to determine how concerning this figure truly is. The issue underscores the complex trade-offs faced in keeping communication safe and open across billions of global users. There are no easy solutions, but Facebook’s willingness to be more publicly accountable as it works to enhance its systems will be critical going forward.
Summary of Key Points
- A leaked Facebook memo estimated up to 300,000 content moderation mistakes occur daily.
- This seems plausible given the billions of posts to review, but lacks context on severity.
- Automated systems and human moderators face challenges around nuance, language, evolving policies and subjectivity.
- Both false positives and negatives impact expression, harm, trust and bias.
- Facebook claims it is improving processes, but transparency is still lacking.
- The overall rate seems low vs. total activity, but harm depends on error types.
- More clarity is needed around Facebook’s mistakes to evaluate their true impacts.