Anything I say would seem condescending, but I'd go on anyways.
But there are way worse lives than being a virgin. Being hungry, being a criminal, being a disappointment to your father, messing up people's lives, being in toxic relationships which cause you PTSD for life, never loving your wife and realizing that when you are 70 and never having the balls to tell her. All these curled up little balls of people you are jealous off might have traumatic pasts and issues which they deal with a partner. Some of them think about them every ten minutes and fidget. There is the insurmountable guilt of damaging a person's soul that they'd never be able to trust someone again fully.
We all like to believe our stories are the worst there is, but trust me a lot of us feel the same way, virgin or not.