Because a skirt cannot give consent

That's almost too simple, nearly to the point of absurdity. Of course like suits, skirts can't talk and verbally given consent, but then 'little black dresses' and the like are often worn deliberately to be alluring and sexually seductive.

Dress is often a signal. No more or less than a flag, but the right or wrong choice of those often meant life and death. Wearing the wrong uniform could get you summarily executed. Maybe it shouldn't then be a big surprise that something like a uniform can be among the least factors that makes the difference between a pleasant evening and being victimized.

encouraging the idea that the skirt is "asking for it" encourages men to ignore the women's mouth which might not be asking for it

What signal (if any) is intended, what's given, how it's received and by whom are all separate things. Not always wanting to dress modestly, people sometimes dress provocatively, but also want to control the reactions they provoke. One is their liberty, the other is not.

One would justifiably think a person a fool who inadvertently walked into a Blood neighborhood wearing Crip colors. Similar to how sometimes very young girls get assaulted drinking with older teens for the first time. In neither case would it be their fault for being assaulted or taken advantage of, but they also both share a substantial amount of responsibility for giving the wrong signals, putting themselves in harms way and being oblivious to the probable consequences. Some people who play with fire get burned.