As I understand his system, private community might have rules who can join or deal with them. Some private communities might have backward views on race or sexuality (Hoppe doesn't see this as backward), so they will also have such rules.

I think what would really happen in Ancapistan is that social pressure and ostracism from wider society will act as counter balance on these private communities (he doesn't mention this effect of his own system as he tries to pander to alt-right).

Physical removal comes only when someone break rules of community stated in advance.

I think a lot of controversy around Hoppe isn't actually about his political system but rather about his cultural views (which I don't share) and the way he interprets his own political system.