Freedom of speech is a protection from the government not from social consequences.

You've got it almost exactly backwards. Freedom of speech is societies legal protection through government of an alienable liberty, the freedom to express ones thoughts, to express oneself. Government is a social consequence. Freedom of speech is precisely the legal protection of individuals by the government from social consequences, from unjust reprisals because of their speech. Like not being jailed for opposing a governments taxation policy (today it just happens to be opposing taxing the internet not tea).

If I'm a neo nazi the government can't do shit but employers and people and the community...

That's just a simple threat; saying people can 'do shit' implying they should and will. In practice you're advocating cowardly group retribution against individuals expressing unpopular opinions, social censorship via the vigilante enforcement of conformity. That's the argument for beating, running people out of town, tarring and feathering, stoning, crosses on lawns, badges, lynching, burning, and necklacing and so forth. Perhaps not surprising coming from a low life hood rat.

Those are the means used to oppose virtually all novel progressive and benificent social developments in history and oppress. The governments guarantying the protection of free of speech is not Pontius Pilate washing their hands of responsability, absolving themselves of the consequences, leaving 'justice' to prejudiced and bloodthirty mobs and fanatics. No matter how vile, dangerous or disturbing those opinions may seem to some. Government is the only shield you have from being torn to shreds or cruxified for mere expression (e.g. Inre). There's no need for protection from mobs and their leading fanatics (e.g. Savanarolla, Sanhedrin, Meletus Lycon and Anytus and so on) for positions they like.

There is no reason to think being <X>... that your community has to accept your views.

Maybe, in many instances probably not, but there's many very good reasons to believe a community should give anyone a fair hearing, especially those whose views you happen to disagree with, even those you may despise, extending as far as cranks and lunatics. Acceptance, not including the number allowed to assemble to listen, where, how and why, is the prerogative of individuals and their conscience, best expressed through ballot boxes. That's simply the basis of a civil society, that's isn't entirely homogeneous. Any and all means of otherwise coercing, guiding and controlling acceptance should be considered extremely prejudicial socially, among the most dangerous domestic threats to the republic, attacking its foundations. Perhaps the ideal the founders were striving for was that politics should be a civil contest of ideas not a barbaric contest of people (who is stronger, most handsome or richest), parties (whose is biggest or best armed), or popularity (may the best idea win, not the most popular).

but employers and people and the community can still have say

Every person in the US has the right, and should have or be given the opportunity, to speak their mind freely without the fear of unjust reprisal. That seems to be just basic freedom.