I'm in the same boat. I used to be pacifist when i was younger. And I think it's worth remembering that the vast majority of non-pacifists don't like violence either. Calling it a "neccessary evil" sounds like such a platitude, but it gets close to the core of the issue I'd say.

Also one has to ask if leaving nazis in peace and shouting slogans will really win hearts and minds. I have my doubts. Sure, confronting them physically will not go over well with everyone. There will be people who think this is rash, unneccessary and potentially dangerous. But I think it's important to considers who these people are. People who critizise antifa groups and so on are (in my experience at least) mostly the people who are not threatened by the fascist groups we oppose. They might be regular guys who obviously want nothing to do with white supremacy and all that stuff, but even if said groups got stronger they wouldn't have to fear for their lives.

But what about people who were victims to right-wing violence in the past? How do they feel if they see uniformed fascists marching through their town without challenge, calling for their death or subjugation? What about people who are afraid to express their sexual orientation, gender identity, religious affiliation or political believes in public, because they fear repercussions? Do they appreciate the prioritasation of the "free speech" of fascists above their own? There are a lot of people whose hearts and minds are won by actions and not by words and in my opinion they are far more valuable for the anarchist community than the white middle class liberals who wrinkle their nose at any attempt at active anti-fascism.