Facebook has over 2.9 billion monthly active users as of the third quarter of 2022, making it the largest social media platform in the world. With so many users, Facebook grapples with various issues like hate speech, bullying, and disinformation campaigns. One of the most persistent problems is fake accounts created to spread spam, scams, or misinformation.
Facebook employs both automated systems and human reviewers to detect and remove fake accounts. Here is an overview of how Facebook handles fake accounts:
Defining fake accounts
Facebook considers an account fake if it meets any of the following criteria:
- The account was created by an automated program or script, known as a bot
- The account pretends to represent a real person but is used for deceptive purposes
- The account is Duplicate or multiple accounts operated by the same person to mislead others
- The account engages in behaviors that circumvent Facebook’s policies, such as financially motivated spamming
So the term “fake account” covers a wide range of malicious and policy-violating behaviors. Facebook aims to quickly find and disable any accounts engaging in these behaviors before they can do harm.
The scale of the problem
Facebook reports removing billions of fake accounts every year:
| Year | Fake accounts removed (approx) |
|---|---|
| 2019 | 5.4 billion |
| 2020 | 5.7 billion |
| 2021 | 7.6 billion |
For context, Facebook had 2.27 billion monthly active accounts at the end of 2019. So the number of fake accounts disabled represented nearly 240% of real monthly active accounts.
This demonstrates the massive scale of fake account creation on Facebook. Attackers are constantly registering new fraudulent accounts faster than Facebook can remove them.
How Facebook detects fake accounts
Facebook employs both automated and manual tools to detect fake accounts:
Automated detection
Facebook designed AI systems that analyze account behavior and patterns to identify policy violations. Some signals the automated systems look for include:
- Suspicious email addresses used during registration
- Suspicious phone numbers used for two-factor authentication
- Accounts registered in quick succession from the same IP address
- Identical or nearly identical profiles Indicatedbot-like or scripted activity
Accounts flagged by these systems get reviewed by Facebook’s content moderators.
Manual review
Facebook employs thousands of content moderators as the human line of defense against fake accounts. Moderators manually review accounts surfaced by users and Facebook’s automated systems.
To determine if an account is fake, moderators investigate for signs like:
- Stock or stolen profile photos
- Little to no personal information
- No posts or basic scripted posts
- Mass friend requests or repeats friend requests
- Suspicious engagement only with Pages or Groups
Moderators can request government IDs or video selfies from suspicious accounts to verify identities. Failing to provide adequate verification leads to account disabling.
How accounts get disabled
If an account gets confirmed as fake through automated or manual review, Facebook disables the account to prevent further abuse. Here is the account disabling process:
Step 1: Account lockout
Facebook instantly locks suspected fake accounts to prevent the owner from logging in or making further posts. Locked accounts may display a notice asking the owner to provide ID verification.
Step 2: Review appeals
Account owners can appeal lockouts if they believe Facebook made a mistake. Appeals go to Facebook’s Community Operations team for review. Valid appeals lead to accounts getting restored.
Step 3: Permanent disabling
If an account stays confirmed as fake after any appeals, Facebook permanently disables it. Disabled accounts get removed from view, and their usernames become available for new accounts.
Step 4: Monitoring for reactivation
Facebook closely monitors disabled accounts to prevent reactivation. Attempts to re-register a fake account under the same name/credentials result in instant disabling.
Limits of Facebook’s detection
Despite Facebook’s efforts, massive volumes of fake accounts continue to slip through the cracks. Challenges Facebook faces include:
False negatives
Facebook’s AI tools still miss many fake accounts exhibiting new patterns not yet incorporated into the detection models.
New registration tactics
Attackers constantly change their registration tactics to mimic real human behavior and evade detection.
Limited moderation reach
With billions of users, Facebook’s content moderators can manually review only a tiny sample of accounts flagged for suspicious behavior.
User reporting gaps
Users often fail to report fake accounts not directly spamming them, allowing indirect threats to stay undetected.
Steps users can take
Users can help Facebook curb fake accounts by:
- Enabling two-factor authentication to secure your own account
- Adjusting privacy settings to share information only with trusted connections
- Reporting suspicious accounts and content to Facebook
- Thinking critically about friend requests and information shared on Facebook
Looking ahead
Reducing fake accounts remains a priority for Facebook. Some emerging initiatives include:
AI detection research
Facebook is expanding its AI research to uncover new signals and patterns that identify fake accounts.
Industry collaboration
Facebook is sharing signals and best practices with other social networks to improve fake account detection across the industry.
Verification program expansion
Facebook is exploring expanding ID verification to more users to make faking identities harder at scale.
Conclusion
Fake accounts pose an enormous challenge for Facebook, given its billions of users. While Facebook deploys extensive measures to find and disable fake accounts, millions still slip through the cracks to propagate harmful content.
As attackers craft increasingly sophisticated fake accounts, Facebook will need to double down on advanced AI systems and collaborative industry solutions. With diligence across its security teams, content moderators, and users, Facebook can mitigate but likely never completely eliminate fake accounts.