You misconstrue its holding. It is very clear.

We are not prepared to hold that a minority, residing or remaining in any city or town where smallpox is prevalent, and enjoying the general protection afforded by an organized local government, may thus defy the will of its constituted authorities, acting in good faith for all, under the legislative sanction of the state. If such be the privilege of a minority, then a like privilege would belong to each individual of the community, and the spectacle would be presented of the welfare and safety of an entire population being subordinated to the notions of a single individual who chooses to remain a part of that population. We are unwilling to hold it to be an element in the liberty secured by the Constitution of the United States that one person, or a minority of persons, residing in any community and enjoying the benefits of its local government, should have the power thus to dominate the majority when supported in their action by the authority of the state. While this court should guard with firmness every right appertaining to life, liberty, or property as secured to the individual by the supreme law of the land, it is of the last importance that it should not invade the domain of local authority except when it is plainly necessary to do so in order to enforce that law. The safety and the health of the people of Massachusetts are, in the first instance, for that commonwealth to guard and protect. They are matters that do not ordinarily concern the national government. So far as they can be reached by any government, they depend, primarily, upon such action as the state, in its wisdom, may take; and we do not perceive that this legislation has invaded any right secured by the Federal Constitution.