Do you want to do this because you feel you are missing out on something or do you want to do it because you feel you should?
It's funny, really. I used to have the same attitude until I couldn't handle the isolation that came with it anymore. This was in my twenties and thirties. My personality changed and now I am a nervous, anxious passive agressive milquetoast niceguy (at at my worst). I really hate it and I sometimes wish I could go back to being the snarky antisocial old_man_trouble of old.
As the top comment says: the first person to be comfortable with is you. And the first step to that could be to stop comparing yourself to others and find a state of congruence, where what you do and think and say fits who you are.