They are not all idiots, nor all or nothing.
The only anti-vax person I have known was not at all against all shots. He was against the quantity and how they were delivered.
The guy I am talking about is without a doubt a genius in many ways, advanced degrees from top engineering schools, numerous patents currently working in production optical networks, he sincerely believed over-vaccinations could cause autism, and he over educated hisself on the subject, he was close to obsessive in his research.
I don’t know other people against vaccines, my kids were older by the time it became a thing, and so I never took a lot of interest in his passion, but he was convinced. (Both his father and two brothers were Doctors, smart family overall)