You are not an asshole. Not at all. OMG I cringed reading this; if my husband did that to me I would lose my friggin mind.

Sit and have a calm talk with your husband about thresholds. Everyone has a different thresholds of tolerance for different activities. Where something might be enjoyable in some quantity, but can quickly become tiresome or even unpleasant in greater quantities.

Ask him hypothetically how he would feel about spending a quiet weekend alone with you. Be good right? How about 2 in a row? Or 3? How long until he becomes restless or bored or antsy for someone else to talk to? Tell him that you know that your limit for being alone together is much higher than his, but that you would never want to inflict that on him, because the courteous thing to do is to recognize when your partner has a different threshold than you do, and respect it.

Ask him some questions about why he feels he needs you to meet these people. He seems to do fine on his own. It's especially strange that he wanted you to come to an event where other wives weren't there. Ask him why he thought that was necessary. Try to get him to unpack his own motivations here, and keep the conversation open.