Well, couple of things.
1) The object identification claims it's all done on-device. When I affirmed that it did, I was going solely on Apple's word. For all we know, it could be sending data to Apple. But I don't think it does.
2) Right now, your iCloud photos are encrypted on Apple's servers, but they have the keys to decrypt them. They could look at them if they wanted to or were compelled to. As far as I know, for as long as iCloud Photo Library has existed, Apple has had this ability.
3) With the exception of (2) above, the photos are generally not decryptable by Apple. To what degree this matters is unknown. Apple (the company) can decrypt your photos, but most people at Apple probably can't (decryption keys like this are usually guarded and only accessible to a very few people in a company.)
But the way they've implemented CSAM scanning is interesting. They've broken a separate (presumably) decryption key into pieces. The pieces (called safety vouchers) individually don't allow for decryption. Once a certain number of vouchers is in the hands of a single party (presumably Apple) then decryption can occur. The things that are encrypted with this key are the hash itself and what they call the "visual derivative" (I'm not 100% clear on what this is.) With this information, they can verify the match and, if relevant, pass it along to the authorities.
This all comes from a whitepaper they've published. https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf
Of course, all this means that you have to trust that the system works the way they say it does. It might not. We can't view the iOS source code. It could be transmitting anything and everything we do on our phones all the time. So could Android. Or your random feature phone from your carrier. At some point you have to decide to trust that what someone is telling you is true, or opt-out of using those products or product categories.
Anyway, all that is why I don't think this is a big deal. Taking for granted that they're being up front and transparent with the details of how this all works, it's a pretty decent system. And it paves the way for actual, real encryption of iCloud photos without Apple having the ability to decrypt them except with sufficient safety vouchers. If that comes to pass, it's a way better system than just about any other cloud-based photo library (because they all essentially do this scanning on their servers.)
For what it's worth, there are a couple of interesting numbers being thrown around. I don't know the source for them. Googling the keywords will yield several sites reporting the same thing, but I haven't found the source.
Those numbers are 265, 20 million, and 500 thousand. Respectively, that's the number of CSAM images reported to NCMEC last year by Apple, Facebook, and Google. I just find that interesting.