Content warning: This post discusses an investigation into the proliferation of child sexual abuse imagery online.
Here’s the high-level overview from reporters Michael H. Keller and Gabriel J.X. Dance:
Pictures of child sexual abuse have long been produced and shared to satisfy twisted adult obsessions. But it has never been like this: Technology companies reported a record 45 million online photos and videos of the abuse last year.
More than a decade ago, when the reported number was less than a million, the proliferation of the explicit imagery had already reached a crisis point. Tech companies, law enforcement agencies and legislators in Washington responded, committing to new measures meant to rein in the scourge. Landmark legislation
passed in 2008.
Yet the explosion in detected content kept growing — exponentially.
As you might expect, the investigation explores where to place blame for the growth of this kind of crime. And soon enough it comes to tech platforms — in particular Facebook Messenger. The reporters write:
While the material, commonly known as child pornography, predates the digital era, smartphone cameras, social media and cloud storage have allowed the images to multiply at an alarming rate. Both recirculated and new images occupy all corners of the internet, including a range of platforms as diverse as Facebook Messenger, Microsoft’s Bing search engine and the storage service Dropbox
Encryption and anonymization can create digital hiding places for perpetrators. Facebook announced in March
plans to encrypt Messenger, which last year was responsible for nearly 12 million of the 18.4 million worldwide reports of child sexual abuse material, according to people familiar with the reports. Reports to the authorities typically contain more than one image, and last year encompassed the record 45 million photos and videos, according to the National Center for Missing and Exploited Children.
In a Twitter thread, Facebook’s former security chief, Alex Stamos, stood up for his old colleagues here: “I’m glad the NY Times
is talking to the incredible people who work on child safety every day,” he wrote
. “One point they seem to be a bit confused about: the companies that report the most [child sexual abuse material] are not the worst actors, but the best.” And indeed, if you talk to NCMEC and other organizations who work on this issue, they’ll tell you that they see tech platforms as essential partners in fighting child predators.
But what if tech platforms weren’t such good partners? And what if the reason was encryption?
It’s a tough debate, and it’s one that we’re about to walk straight into the middle of. The reason is Facebook’s plan to encrypt its core messaging apps — Messenger and WhatsApp — by default. The effect of the move on law enforcement’s ability to fight crime is unknown, but certain to be controversial.
I find the fears to be straightforward and rational. Today, thanks to Facebook’s efforts in particular, law enforcement detects millions of cases in which terrible images are being shared around the world. In thousands of cases a year, according to an event I recently attended at Stanford about encryption
, this leads to arrests of the perpetrators. But if you were to shield all those messages using encryption, the argument goes, you would essentially be turning a blind eye to a disturbing and growing problem.
It may simply be impossible to moderate
the content that is exchanged between all of those people. But maybe there’s a simpler, blunter approach. We take for granted that you can send images, links, and videos on Messenger, but what if you… couldn’t? What if we’ve gotten the cost-benefit of being able to send a video on such a large, central platform wrong? Messenger could simply be text-based, as old messaging services were: Easier to moderate automatically, and without the risk of harmful videos or images being distributed. There’s an even stronger argument that the same calculus might be applied to Live videos on Facebook, which have previously allowed people to broadcast shooting rampages and suicides. True, some users would go elsewhere, the content would persist in some fashion, but it would not be supported by the dominant social network. There is a chance, at least, that its creation and distribution would be impeded in some way, especially if other companies followed suit.
I’m sure the idea of banning all link- and image-sharing in Messenger will find favor in, for example, authoritarian governments. Just imagine the nettlesome dissent that gets spread via links and images! And yet it also seems notable that not even Russia or China have taken such an extreme step — they have instead ramped up their dystopian surveillance operations in an effort to root out dissent at the source.
Evil folks will always be able to figure out the most efficient way to be evil. The question, though, is how much friction do we want to introduce into the process? Do we want to make it the default that the most user-friendly way to discover your “community”, particularly if that community entails the sexual abuse of children, is by default encrypted? Or is it better that at least some modicum of effort — and thus some chance that perpetrators will either screw up or give up — be necessary?
To take this full circle, I find those 12 million Facebook reports to be something worth celebrating, and preserving. But, if Zuckerberg follows through with his “Privacy-Focused Vision for Social Networking”, the opposite will occur.
To state the obvious: the trade-offs involved in the discussion of encryption vs. security are agonizing. It’s easy to defend encryption in the context of most private discussions between adults, whether it’s dissent against the government or of a more personal nature. It’s much harder to defend encryption when it’s being used to share images of child abuse, or to plan terrorist acts. And we lack easy methods for balancing the risks versus the benefits. How much freedom does an encrypted messaging platform have to support, to make up for the terrorism that it might contribute to? How do you design that test?
One way we can approach the problem is by thinking about it in terms of internet problems versus platform problems. As I wrote earlier this year:
Internet problems include the issues that stem from the existence of a free and open network connecting all of humanity together. The existence of forums that allow white supremacists to meet, recruit new believers, and coordinate terrorist attacks is an internet problem. The proliferation of free file-sharing sites that allow users to post copies of gruesome videos is an internet problem. The rush of some tabloids to publish their own clips of the shooting
, or analyze the alleged killer’s manifesto, are an internet problem.
Viewed this way, I see the spread of child abuse imagery online as much more of an internet problem than it is a platform problem. It’s true that platforms provide an easy way to disseminate this content — but it’s also true that predators have many, many alternatives to Messenger, and actively use them. I’ll never forget the shudder of a person who used to work at the Tor Project
when they told me that a meaningful percentage of the site’s users at any given time appeared to be actively engaged in sharing child abuse imagery.
And that’s to say nothing of the other big platforms where child abuse imagery lives. These files exist and are transmitted on iOS, Android, Mac, and Windows, to name four big ones. Should we compel those platforms to scan user screens periodically and check them against hash lists of known child abuse imagery? It’s possible to do that without involving the encryption debate at all — users’ screens aren’t encrypted. Does that make it a better idea, or a worse one?
Child abuse imagery is an internet problem because it’s fundamentally about how the friction involved in bad people meeting one another, and enacting awful schemes, has now dropped to zero. You could close every big tech company on earth and, assuming the the TCP/IP protocol still existed, still find that child abuse imagery was spreading around the world.
In the meantime — happily — it’s an internet problem that tech platforms have worked actively to solve. I’m sure they could work harder and do more, but it’s notable that at a time when people hate platforms for almost everything, the people closest to the subject — the FBI and NCMEC, to name two — seem genuinely pleased with the partnerships they have. It might not be possible to ramp these efforts up, or even preserve them as is, in a world where encrypted communications are the default.
But it’s also worth trying. These images will continue to proliferate around the internet regardless which platforms are currently dominant. To focus narrowly on the question of how they are transmitted lets a great many people — and companies — off the hook. A solution that preserves encryption while automatically checking shared images or links for connections to known child abuse imagery and reporting it to law enforcement might not be possible. But before we give up on the idea of private communication online, we ought to look for one.