I agree with this sentiment. Choosing to go into math set me up for isolation and quite a bit of harassment from men who would declare themselves "nerds," are some of the most sexist pieces of shit I have ever met, and, yet, actually believe themselves to be more enlightened and fair when it comes to gender (Dunning Kruger Effect, no doubt). These people make up a small minority... maybe 20% in my department. But that's almost double the women where I am (where gender ratio is below the national average).

I think the problem can be abstracted even further, though. It's people who are overconfident in their intellectual ability, operating on insufficient knowledge, and convinced that they are completely objective. These are the kinds of people who go with their gut reaction and then reason after the fact. Yet, believe they came to their opinion in the reverse---that reason led them to their conclusion. Their overconfidence prevents them from assessing that reasoning with any kind of rigor, collecting more information on a topic, or even trying to understand the arguments of the opposing side. So, when they meet a woman in math, they're instantly attracted to her because, given the ratios, they're a bit desperate (nothing inherently wrong with that). When they are rejected, they feel resentment and wish for that woman to be gone. Then they reason that she has no business being there. Other men join in because, frankly, math is fucking hard and destroys your ego, so some benefit from being able to say, "She doesn't belong here (therefore, I do)." Now the harassment starts. And if she or others challenge this behavior, they'll insist that they're being objective, they'll bring up averages, they'll dismiss, they'll say she's being hysterical. They'll make a sexist joke.

Honestly, being in math has made me blind to some kinds of sexual harassment because it feels normal now. Almost inevitable: Of course men would joke that women are only good for cooking and cleaning. Of course men would joke that women can be right "on the rare occasion" because even a broken clock is right twice a day. Of course men would laugh about a male graduate student "dating" one of his students and then finding out that she immediately stop returning his calls and dropped out of school. That's funny; nothing unusual or disturbing about that. Of course a bunch of men would joke about sexually assaulting one of their female colleagues. That's so funny; a group of guys joking like that in front of her and then taking her picture so they could remember her running out of the room in horror. Of course it's funny when a professor, upon being asked for an appointment to discuss a particular calculation in Lie Theory (the professor refuses to hold office hours), tells the class the next day, "Someone asked me something about this calculation--I don't remember who--whoever it was, just read the damn book!" That's hilarious; she's the only woman in that room, she had to work alone as it is, and, oh yeah, she did read the book and it didn't discuss that calculation in full generality. The problem is these women. These women just can't take a joke. If women drop out, it's because they don't have the intellectual capability. Something, something, variability hypothesis, something, something.

The lack of social skills just makes some of the dickishness more transparent. That's the problem with nerd culture in math. I would also take autism off the table. I think it's implicated because the percentage of people with autism in math is so high, but I don't think it's exactly correlated. I think it has more to do with fragile egos, the need for some kind of reinforcement (either positive reinforcement via praise or something competitive--needing a group to say you're better than them), and a lack of maturity.