Interesting. I'd never actually thought of it like that. I would imagine that a majority of the land used would be taken up to build housing. Are we proposing organic plant based diets? If not then I would have some concerns over the exponential increase in pesticide usage. It destroys wildlife at all levels. I would also imagine that there would be a lack of animal based fertilisers so that could be a potential issue.
I've always wondered how it would impact other countries. Those that don't have the climate to grow fruit and veg. Is a plant based diet more of a Western privilege because we're fortunate to live in a climate that allows for plant growth.