The stress I'm referring to, which perhaps you don't share (in which case you're a bigger person than I am), is the pure uncomfortableness of knowing that other people look down on me for one reason or another, or hold opinions I find repugnant.
Oh, I share it. One of the feelings that I think makes it uncomfortable is the feeling that they're still a danger to your beliefs. (It also brings up the issue for both you and them of how nice can you be to someone if you secretly find them repugnant? Maybe you won't always be able to hold back a scoff or a dirty look if they say something that mentions their beliefs.)
In some sense, being around people you disagree with in a core way re-enforces a fear that you are alone, and the world is on the side of your opponents. I think that's one of the reasons people hate animal violence in movies more than human violence. We all know human violence is awful but some movies actually do have animals getting abused in their movies. It's not as out there. The racist/sexist joke is a lot more likely to have sincerity in it than the joke about beating nuns to death with orphan babies. 'Cause the latter is so absurd it's more of a coping mechanism for good people to deal with the existence of evil than anything else.
The animal advocate is already prone to the view that the world sees as animals as emotionless automatons to be treated like furniture, and a hunter who thinks violence against animals is fun only reminds them of their worst feelings about the world.
In that way, I don't think a white nationalist is a good example of what we're talking. Unless civil rights is a particular passion of yours beyond just agreeing with it and moving on, the example feels very impersonal, white nationalists are kind of just people everyone agrees are bad without any personal stakes in the matter so they can't feel threatened by their existence.
If you've ever felt uncomfortable around very religious people who have been nothing but polite to you, you might have a sense of what I'm talking about.