You have a right to feel bad about what happened. He had a choice on how to handle himself in the situation and it sounds like he didn't just tell you that a joke wasn't funny but instead went on to berate you. He didn't have to use that tone but he did. He didn't have to be misogynistic say all women suck as comedians.

If you have these kinds of interactions with him and you are afraid of talking to him because he will make you feel worse... I hate to give you this advice so bluntly, but I think you should move on from what sounds like a toxic relationship.

You are 21, do not waste your youth on a loser who makes you feel so small that you have no voice.