It's not rude. And it's certainly a fair thing to ask. And a good partner would consider what you need in terms of safety and comfort before wanting to proceed.
Let me try to take a step into a fairly normal / logical reason someone might react this way. First and most simply, he could have been looking for a hookup not a relationship - and you just ruined how easily that was going to happen - perhaps he was playing the boyfriend type and you just made it dramatically harder. Smash after third date, right? ANNNND you just pushed that out a few weeks now he's frustrated.
Second and probably more likely... I think guys react badly to this question because of the splinter it creates in their mind. There's now this thought that if this is in your head as a need, it's because you've had a scare in the past ~ and now the other person is worried about your past both in terms of STD risk and the perceived experience (quality/quality) of partners you've had in the past. Especially for men, we're competitive creatures. We want to feel like we're the best you've ever had. It can honestly be hard for some to date in the modern world because we know we're not the only one. It takes a lot of growth to get comfortable with this idea, and while most men get there, quite a few (notably many who are single above a certain age hold onto this still). It's not at all realistic to imagine we're the best, but it's a lie we love to pretend is true. ANYTHING you do that takes the shine off that armor... weakens the relationship and the passion. Even a casual mention. Not saying it's fair, probably a little petty and insecure. But sadly it's true for many. So you probably just triggered this guy by even having it as a concern.
Let me repeat, I'm not defending the mentality, just offering a hypothesis on where it might comes from. And ladies, unless you're trying to unnecessarily damage your man, allow him to believe the myth. If he's smart enough to not ask, be smart enough to let him believe. Most of the time I think you'll get a happier, more engaged lover... who may not be the best, but he'll be the best version of who he CAN be. And that's all you can really ask for.