This is all written in a confusing way, I would encourage you to re-read this. There are so many points made here that need clarification its hard to know where to begin.

You never mention how old he is, which is relevant for background information and clarification.The confusion here is on a level where you imply at one point that it is him that raped you when you say "I replied with something like I had sex with you when I didn't want to...".

The belief that all people should be believed when they proclaim abuse, is a bit dubious in my opinion, as it disallows for the appropriate estimation of evidence and is akin to blind faith, and is just logically problematic in general, though I understand why its important for real victims psychologically. In this case, again if what you say is all true regardless of its convoluted nature, then he sounds like he has emotional issues to say the least.

Your marriage was never healthy in the first place for multiple reasons, largely revolving around his unwillingness to have sex with you, his desire to sleep with other women, and your non mutual participation in said "open relationship".

You say that sex to you is "love", but there is so many problematic aspects of saying that, including that if he was sleeping with other people while married to you, due to the "open relationship" status, then he was in love with all those other women, and not you. This is a really twisted point of view in my opinion. Sex can only ever be an expression of love, and it is clearly not always so. It is my assumption that women feel a great deal more physiologically when participating in fulfilling sex that could lead to this association with meaningful love. But this is quite illusory, the male does not feel nearly as much during sex, so be careful when becoming attached to someone solely on the premise that you felt a great deal of physical pleasure when you had intercourse. Love is something that transcends the physical, it encompasses the mental, and the practical.

Don't allow yourself to get lured in to another abusive relationship, there are plenty of guys out there who just want to be nice to you and love you. Use this as a learned lesson to recognize the signs of someone you need to avoid in future relationships. This requires a lot of self reflection on why you continued to put effort into the relationship and taking ownership of your share of the responsibility in it, which sounds like you are more than capable of doing, so you have everything to look forward to. Also realize you don't need to be in a relationship at every single point in your life, allow for alone time, and just you time. Self improvement is something that you are responsible for and no one else can give you, not even a lover. Wait for the right person, don't go falling head over heals for just any ol' guy, you've seen where that can lead to. Trust is everything, and you have to have that above all else, even love.