authoritarian countries will ask them to report anti-government files

Except that’s not how this tech works, at all.

It can’t search for content or scan images for subjects. It can’t even recognize actual CSAM unless it exists in the hash database, and even then, it has to be the same image.

For context, I used to work at a social media platform that unfortunately had a lot of CSAM slip through. The sad reality is that people that deal in this stuff collect and trade it like baseball cards. So, the way this tech is designed, is that it hashes the known images (read: evidence submitted in investigations that resolved in guilty verdicts). When images are passed through it, it’s checked against the hashes.

It’s designed like this because platforms like Facebook and Google Photos, or whatever it was called back then, needed to check millions of images a second without putting strain on their existing systems.

So let’s say, a government wanted to start flagging people with images of a certain book. They would have to add hashes of that image to the CSAM database, but then they’d only catch people who had the exact same image. It can spot variations of the image, like if it’s dropped, colored, or rotated, but it wouldn’t be able to match a photo of a person holding the book up in a photo or even a photo of the book itself.

Does that make sense?

I’m all for skepticism around any tech that insists on looking at your data. But understand that this is basically the same method as reading EXIF data for GPS coordinates. It’s very rigid, which is why it’s got such a nominal margin of error.