Even though you feel that way about yourself, if he really matters to you, you NEED to spend time with his friends, at least once a month or so. Yes, I get that your social anxiety makes you not want to be around people, but no one can completely avoid being around them (hence you spend time with your b/f).

I know what it's like to feel nervous and self-conscious, and I would not expect you to break your comfort zone daily or weekly. But is he worth the effort? Or do you want to be a hermit?

A relationship is a two-way street where both people occasionally must evacuate their own comfort zone in lieu of being real partners. Otherwise, there's no point because then life is way too easy. Also note there is a saying "Do something that scares you every day". If you cannot meet him just a little bit, there is no future for you and you may end up alone like older people with SA do. People who didn't have a bf who wanted them to meet their friends but instead had people not care they exist until they broadened their SA.

PS If you spend time with them, you do not have to mention the uni thing. Talk about anything else, avoid the subject if you must. He's not asking you to feel like a loser (I hope) but spend time with his buddies. This is the sign of a man who cares.