I do not at all think that transpeople should be barred from the bathroom's of the gender they identify as. But I will argue that there are good reasons to keep public bathrooms gender assigned.
A sexual predator is not going to be deterred by a law that says they can't be there.
If a guy wants to creep around the women's bathroom he will obviously stand out and is clearly breaking a norm by being there. A creep can easily be send away. But if the bathrooms are gender neutral he can creep pretty freely without people being able to send him away. He has a right to be in that bathroom. A creep will still act creepy but he's easier to identify as a creep and easier to get rid of. I've been in quite a few bathrooms (usually gas stations) with warnings about men trying to hide in there to creep.