The happiness you get from rejecting someone and watching them get hurt comes from anger and hate.

No it doesn't, it comes from the satisfaction of knowing that things are as they should be. If I see an idiot driving like a lunatic, and they crash, that's as it should be, and it's satisfying to know. All the OP is doing is watching them crash, and knowing that's how it should be.

Wouldn't you rather see a woman get hurt, than see a man suffer his life away, for her idle comfort, and indulgence?

It's natural for a creature to want to expel a parasite, and it's also natural to take pleasure in it being expelled; one should have no sympathy for a tapeworm, or someone who acts like one.

You can hold on to these feelings all you want, but they do lead to self-destructive behavior

Finding one's own personal freedom, often comes at a price, and those growing pains are only temporary. It's far healthier to express oneself through a situation, than it is to stand back in fear.