I wouldnt be too bothered by the fact he wouldnt be willing to also have sex with other women, I think it's different if it's the same sex. Would he be okay with you having sex with another man? My guess is probably not, because he is your man and should be enough to satisfy your needs and wants from a Male body you know? So he doesnt want other women because he already has a woman. If he was into guys, he might feel different about being willing to sleep with a guy.

Maybe give it some time and have another conversation with him just to make sure his mind hasn't changed. If he still says you should go for it then go for it! He's probably confident enough in your relationship that he knows you guys can handle the new arrangement.

Maybe down the road he'd be open to threesomes, it might not feel like cheating for him if you're there participating too