I feel similar in that I too would prefer a partner that only had one past partner but it doesn't bother me that much if they were a bit more. However I think there is an upper limit that would make me less comfortable tbh but it depends on how they are overall. I would be less comfortable if they were casual partners. Mine was a long-term relationship so while it was one partner it may have been for a longer period of time than average. I suggest you just learn to overlook it if possible, especially since you did do it too. If it's not possible and something that you'll get hung up on perhaps you should break it off and stop wasting his time. It could bug you a lot later on and when you have little spats it will come out, especially if you're jealous of his past partners for any reason.