Reading through the comments, I've seen a few different arguments about why this isn't right.
First of all, the New York Times article that has been cited a number of times regarding the same treatment for Muslims, shows their complete bias and anti-Semitism. If it is so bad for Hassidic Jews to utilize this rule, how can they justify supporting it by Muslims? Just shows how much of a piece of shit NYT is.
I don't know enough about the law to completely justify it legally. However, I don't think we should classify it as a completely religious issue. It might have started because of religious needs, but I don't think it should be classified as a religious benefit, because it can help others as well. Like many people said, there are many non religious women who would probably love the idea of gender separated swim. This is an idea which benefits a much larger demographic than just Hassidic Jews and Muslims. If adult swim time benefited some religion, would people consider it an issue?
Secondly, the reason that Hassidic women want separate swimming, is because there is a religious value called "tzniut" or modesty. Hassidic women believe that they must display an added level of modesty around men, because men are naturally attracted to them, and might be tempted to have lewd thoughts towards women. Is that not the same exact way that non religious women would benefit from separate swimming? Because they don't want men ogling them? Why should there be a problem with women wanting some modesty and privacy just because it is a value that their religion teaches?
Thirdly, I agree that men should also have the right to men only swimming, but only if there is a demand for it. In are situation, these women clearly spoke out and requested this accommodation. If there are very few men who would make use of this, there is no need. Just because women have, and make good use of, this accommodation, doesn't mean it automatically has to apply to men too. If there is a significant number of men who would benefit from this accommodation, then 100% they should be given it. But only if that is the case. It is not gender discrimination to make this accommodation for only women, if it is something that men don't need.
Lastly, does it really inconvenience you that much for women to have a little time to themselves? Can't you just change your schedule a bit, so that you can do something else that small amount of time that you can't be in the pool? It honestly isn't that big of a deal for women to have their time- you can go to the pool at any other time during the day. Be a little bit considerate.
If I haven't covered your argument yet, then you are probably a perverted SOB who goes to the pool so he can just stare at women. However, if you aren't, and still have something to say, feel free to reply.
Thanks for reading.