My husband and I were high school sweethearts, and we were each other's first sex partners. We actually have a lot of friends who are in the same situation. We have been together for 30 years, so I have a lot of personal experience to draw from on this topic.
I'm sure someone has already said that you can desire to have sex with another person, but still be OK with being monogamous. Monogamy is a choice. I'm sure he will wish (in a fantasy way) to fuck other women, but that's not a negative on you, or your relationship - it's just the natural urge when there are attractive women. Doesn't mean it will cause him to feel resentful that he can't - it just means that desire is there, which doesn't go away because he's monogamous. Understanding this is pretty important to not feeling like there is something wrong with you, because he's going to find other women sexually desirable.
But, it can also happen to you, too. You might not think you'll feel that way in another decade or two, but you might. Studies have shown that women often lose their desire for sex with the same partner in a long-term relationship, and their desire returns for a new partner.
He (and I) currently really enjoy our sex life so I wouldn't say it has happened yet but I'm scared it might actually happen eventually.
As someone who has been with the same guy for longer than you've probably been alive, I can say that there are so many things that factor into a marriage that worrying about this one aspect is ignoring all the other things that can go wrong. The best thing you can do is to learn to really communicate with each other, because when you have problems, communication is going to be key to getting through them. That applies to sex, money, children, careers, etc.
Marriages change over the years, and how you feel about your sex life, and how he feels about your sex life, is very likely to change. A surprising number of people I know in our suburb have opened up their marriage, because they know that their relationship is solid yet both have desires to have sex with others.
So, my point: you can't predict how you, or he, will feel in the future. I wouldn't get bogged down with this, because so much will change over the years. Do what feels right now, and then deal with this later, if it changes. This isn't like saving money, where what you do now can change the future dramatically, and even if you let him have a "hall pass," you have no idea if that would be better or worse for your relationship, or if it would make a difference in the future.