Hi! I'm a lady who has some dude friends, a couple whom I have a romantic past with. Here is my two cents:
You don't owe a casual partner explanations for your past. I understand why some people feel that you should tell this sensitive information ASAP, but if your partner met your friend group before y'all were exclusive, exactly when would that have happened?
You are perfectly entitled to be friends with your exes.
Your partner is perfectly entitled to be upset about the circumstances. Although, ultimately, I hope he realizes that his feelings - while valid - do not automatically make your decision about your friendships immoral, and that you are not required to tiptoe around his feelings.
Honesty is good for relationships.
What I have found to be a respectful and melodrama-free way to bring up the fact that I've been intimate with people my partner will see is:
1) wait for the relationship to actually form (there's no point in this conversation, for me, if the relationship is casual or short-term),
2) bite the bullet and disclose first (it's awkward, just do it), and
3) give them time or space, if needed, to process their feelings.
My friendships with my exes are very obviously platonic - that and my being upfront/respectful about the issue (in due time) makes it a non-issue.