This seems very black and white, like you two lead entirely separate lives outside of when you're alone together. Do you have any mutual friends? What happens when they're invited over and you all hang out together? And if that never happens, why not because it's weird for a couple to not have some mutual friends. You're NTA, at least as you've described the situation. Sounds like he has his buddies over and you bow out to some other place and the same happens when your friends come over. But if you're hanging out with him and "his friends" a lot when they're over, then you should help the prep and clean-up because you're involved in entertaining them just as much as he is.