I don't know if you fully understand what most vegetarians/vegans actually eat, dw I encounter it all the time (I think the veg prefix is a bit misleading).

I'd say most people replace meat with a nutrionally comparative food eg. tofu, egg, faux meat, beans. We're not exclusively eating veggies (although evidence does suggest vegetarians/vegans statistically consume more vegetables per capita).

So to compare the cheapest chicken I could find:

250g Chicken Wings: $6 250g Chicken Breast: $10

With

200g Marinated Tofu: $4 300g Eggs: $4.40 420g Kidney Beans: $0.8

So unless you're eating those expensive fake meat patties at every meal, a vegetarian diet and some vegan diets are going to be cheaper.