Woman here lurking! I want to chime in with my thoughts. Sorry if that's against the rules or anything, you can remove my comment if so.
I used to work at a place that changed their policy to allow both genders to use whichever bathroom they feel comfortable in.
This happened maybe 3 or 4 years ago. I'm not sure exactly when, but it was around the time that advocacy for Trans rights was at the forefront of the LGBT movement. And so, in support of that movement, my old company implemented this policy. They actually gave us a script to rattle off, if anyone ever asked about it, and we were forbidden from deviating from that.
So anyway, at that job I was a front end supervisor. I went around and if any cashier had a difficilt issue, i would help them with it. Mostly it was customer complaints they didn't know how to deal with. And boy oh boy, did i hear a lot from those cashiers about the new bathroom policy. Lol.
One time, one of the employees, an older lady in her 60s, hurriedly pulled me aside and said "on_too_much_adderall! A MAN just went into the ladies' restroom! You have to do something!!!" All i could do was say to her "i am sorry if you're uncomfortable with it, but as per the new policy I'm not allowed to do anything about it."
My point is i pretended to be apologetic because i was required to - and, many customers and cashiers (mostly women) were uncomfortable with it, but honestly, I didn't care. I didnt mind the policy at all actually. Hell, if someone identifies as a woman and wants to use the ladies room...let them. As long as they weren't being creepy or anything, it was okay with me for a man to use the women's restroom.
And as for women using men's restrooms... I've totally done this because of the fucking lines. Can you really fault me for that? I'd use a stall, and it isnt like I'd drop my pants in front of you or stare at your dick. Trust me. I simply didnt want to wait 30min to use the fucking restroom. And let me point out ive only ever done this in an emergency.
Bottom line is: i think people should just relax when shit like that happens. If the person isn't being a fucking creep, let them do them. Just my opinion.