I was never really concerned with being in any relationship and was more focused on hanging out with my few close friends.
Me too. I got into a relationship when I was 16, but it only lasted a few months. I didn't get into another relationship until early 20s, but I did go on some dates. Teenage girls are crazy! At least those who I knew. The gf at the time would sometimes decline my invitation to hang out, then follow me to make sure I wasn't cheating. Crazy!
It's also probably hard for me to form a relationship because I'm slow to warm up to people and don't like to talk much.
All in good time. Your social skills will change/evolve as you experience more. Also, some women are into slow warm ups.
I feel like my self esteem is hurt and I don't want to get hurt.
Here is the thing about self-esteem. It shouldn't have anything to do with other people's opinion (though it can be difficult to separate). It has everything to do with how you regard your own self-worth. The great part about self-esteem is you get to choose how you think or feel about yourself. So there is little reason not to choose to feel positive about your self. Once you make that choice, behave and evolve so your lifestyle/actions/thinking/etc reinforce those positive feelings.
When self-esteem is not your own choice, you allow other people's opinions dictate how you feel about yourself. Your "self-esteem" is no longer "self"-esteem and it becomes "other-peoples-esteem-of-you-that-you-believe"-esteem. Then you can get into an up and down cycle of how you feel; which leads to feeling unstable about life and stuff. Or if you are around the wrong people they can incorrectly make you feel like shit about yourself - no thank you. Or conversely, artificially inflated and when they go away you can get deflated - no thank you, as well.
The last part about other people's opinions... Those opinions are generally base on a partial knowledge or experience of you. Which means their opinion may not be completely accurate. This means that when listening to those opinions, consider the source when you decide how much significance the opinion should mean to you, and how true the opinion is.
edit: format and corrections