Can I play devil's advocate here?

Telling me how not having sex is frustrating

So he's communicating to you that not having sex is frustrating. How long has your rough patch been going on? Is sex a key factor in your rough patch? When was the last time you two had sex? Does he feel like the dead bedroom is contributing to the strain on the relationship?

constantly telling me to take my clothes off in front of him, pushing himself on me, insisting, making me kiss him, pushing me against corners, etc.

Is it possible he thinks doing these things might spark some sexual interest from you? Have you been receptive to dominant sexual behavior in the past?

I tell him that he is crossing a boundary and that I find this behavior abusive. Every time I bring up the word 'abuse' he gets very defensive and angry. I understand how it can bother someone to be labeled abusive, but he then tells me how I'm overreacting and how it's my fault because I make him upset.

If you are explicitly telling him you do not like what he's doing, and he continues his actions anyway, then yes, this is abusive. However, if he is stopping when you tell him to stop, I wouldn't be so quick to label him abusive. That's really an important disctinction. Being defensive after being called abusive does not necessarily mean anything. Any normal person would feel the need to defend themselves if they were being called abusive by their SO. But if he crosses a very obvious line after you make it very clear where the line is, then you should be concerned.

It's possible his passes at you are his (failed) attempts to cure the dead bedroom. If he never actually crosses a boundary line after you've made it clear where that boundary line is, then check out /r/deadbedrooms