There's really nothing wrong with him being attracted to younger women and his comment seems like a joke. That said, it's perfectly reasonable for you to not feel comfortable with a joke like that. Personally I wouldn't bring up the age thing but I would point out that some of his comments are sexist as to me that's a bigger issue than his attraction to younger women. Obviously I don't know what other comments he makes that references his attraction to younger women but the payment thing is sexist and if he has any respect for women cutting down on sexist comments may help cut down on those sort of comments as well. So yeah, feel free to let him know that you're bothered by some of the comments he makes but unless you know others are bothered by it I'd leave the age thing alone and focus on the sexism.