Here's my problem with this argument, there's two ways to characterise the content of the memo:
The fact that the author starts off saying it's not "anti-diversity" doesn't really mean anything if the content doesn't support that claim. It's like saying "I'm not racist, but ...".
And the fact that the memo includes some good points doesn't exclude the huge amount of biased claims and ideas. When I read it, it came off as incredibly sexist to me, and I had trouble interpreting it anything other than being against diversity, despite the claims to the contrary. Ultimately if you want to publish something like this for everyone to read then you have to accept that you don't get to decide how people feel about it. If you say "I'm not sexist, but..." and they say blatantly sexists things, don't be surprised if people think you might be sexist.
Not to mention that any sane person would have to realize that the chances of alienating lots of people at the place you work might not be the best way to try and accomplish the goal of reducing discrimination.