I'm a woman who has both been raped and falsely accused of rape. So I actually do understand the fear of a false accusation, since I've lived through one.
The accusation I lived through was sad and it hurt me a lot, but it was still a fluke. A teenage girl who had been caught masturbating by strict parents panicked and then said I had been raping her, since I was the last person she talked to that day. It sucked and even though she quickly recanted, by that time, no one wanted to believe the truth. So, yes. I lost all my childhood friends and most of my family over this.
The difference between me and you, though, is that where this is my reality, this is your boogyman story. I have lived in three different countries, have known many people, but I am still the only person I've ever known who has been falsely accused. On the other hand, I know far, far more people (of both genders) who have suffered sexual assault.
The crime of sexual assault is so often minimized. 'What were you wearing?' 'Were you by yourself?' 'Were you drinking?' Every question created to take the burden of the assault from the perpetrator and, instead, place it on the survivor.
Now, look. If a 'decent young guy' is actually decent: doesn't pressure someone for sex, actually respects the idea of enthusiastic consent, a false accusation is incredibly unlikely to happen.
On the other hand, if I take my eyes off my drink -- even if it's in my hand -- well, I have actually had someone slip something into it before. (A coworker at the restaurant I worked.) Walk alone at night? Be happy if you only get catcalls or someone pulling up beside you and asking if you want a ride.
You go on a date with someone that simply doesn't work out and then, go ahead, deal with their rage when you don't wanna have sex. Get so scared they might hurt you that I know many people actually end up giving a blow job to try to get out of there safely.
I was falsely accused of rape when I was 24. And it sucked.
But I've been groped, assaulted, and raped since I was 8. The sexual assaults, dude. That was life shattering.