Regarding the work/friend comments, he says these girls don't mean anything to him, all guys think like that and this is just harmless flirting.

Look, my bf and I regularly comment other people (female and male). I think it's okay to tell friends or even strangers at some point that they have a great taste in clothes, that they've got a nice hair cut etc. However, these comments are not okay:

"I bet your arms could look like that if you worked out"

He's also made comments about how much hotter I'd look if I had a boob job. He said he finds me attractive, I'd just look hotter with bigger boobs.

Seriously, these comments are insensitive and show what kind of person your bf is. Would he say something like that to other people? Even very confident people would feel bothered by such comments.

He says he's being honest and open with me, just having fun, telling me about dresses he wants to see me in

If he isn't happy about how you look, tough luck. Time to find a new SO. You're not his project. And the being "honest and open" doesn't mean he has the right to be a jerk.

everyone would look better if they worked out so he tells me that, and bigger boobs are just in general hotter.

Hahaha, wtf? That's just his opinion. What about the other billions of people around the world? Again, your bf is a jerk.

He's says he finds me attractive and I'm taking these things personally.

No joke, your bf makes regularly fun of you and tells you what he doesn't like about you.... What else should you do? Exactly, dump him and ignore him.

Am I overreacting? How would other girls react? Guys, do you do this to your SOs and if so, how do they react?

You're not overreacting. If I were you, I'd dump this piece of shit really fast. You deserve much better than this guy. And no, my SO would never do something like that. He can appreciate other people's beauty/assets without comparing me to them or talking down to me.

Tl:dr: Dump this jerk's ass and move on.