I don’t see any context here, I mean if you were having the classic “what’s your usual type” discussion, I don’t see anything wrong with him giving you an honest answer... I’m naturally attracted to dark haired men, and my blonde haired husband is naturally attracted to large breasted women (of which I am not) but that doesn’t mean we aren’t attracted to each other... I think there’s an argument to be made that NOT being someone’s “type” means you have stronger chemistry because there was someone more initially drawing you together.
On the other hand, if he just casually mentions this to you out of nowhere, he could be hinting that he would
like for you to get in shape. Which is completely unfair considering you haven’t drastically changed since you got together- he started dating you with your current figure, he shouldn’t expect that to change for his tastes