Homo erectus had a max height of 5'4"(and that's for males....) according to a 2010 study. The famous Lucy was only 3'7".

The fact that modern humans have a way higher max height than our ancestors is proof that evolution preferred taller individuals over shorter ones.

mobility

What do you mean mobility? Mobility for what? Against what? Taller people run faster since they take longer strides. Usain Bolt is 6'5". Women are shorter than men but men still have faster reaction times so that can't be it neither.

being able to survive on less food

Humans are highly proficient omnivores. Do you really think that was a problem for us? African tribes still hunt lions for fun. It's obvious we are alpha predators. We can afford to eat more food. You could also argue that being tall was proof of being well-fed (because if you could sustain yourself at such size must mean you are good at hunting/nutrition) and thus an attraction point for females.

being better suited to climbing.

We are ground dwelling bipedal animals. Swimming and running is more important to us as a species and tall people are better at both of these.

then it would be rare to be short,

You don't see many 5'0 males do you? Because that's how large the average Homo erectus male was.