I feel you. It's especially bad where I live in the UK where there's such a heavy drinking culture that... well, it's referred to as a drinking culture.
I've been dead-set on never drinking from an early age. I was (and still am) a naturally quiet and reserved person, so growing up around family who drink every weekend (and most weekdays) and end up obnoxiously shouting and laughing the entire night long made me feel intensely uncomfortable. Even when I'm hanging out with my closest friends, I either leave or get extremely quiet the moment the alcohol starts coming out; I hate what it does to people.
It's also so tedious having to justify why I don't drink every time the topic comes up, as though it's some sort of crime not to. I don't have anything against people that drink (if I did, I would talk to basically no one), but I hate alcohol and it's so sad how it's become such a cornerstone of socialising.