Let me chirp in here if I may...
Absent fathers have been the cause of men becoming disassociated with their masculinity and male friends for years. At base level interactions with people, you've learnt social cues and conversation techniques that women find easy to communicate with and so do you. So it's easier to connect with women. You probably didn't get many positive man to man interactions I'm guessing, and in my opinion its important to have close male companionship.
That's okay, because guys are actually pretty simple and usually friendly. You just have to DO WHAT YOU LIKE. Literally try things you like and be passionate about doing it. Join groups with people who also like to do those things. You'll eventually realize that connecting with men is REALLY easy if you're both connect on something. Don't take shit from anybody, workout and be honest about your mental and physical health.
People will respect those passions and commitment and you will no longer be worrying about connecting with only males, you'll be connecting with everyone!
There's a lot written about this stuff.
I would suggest reading No More Mr Nice Guy by Rollo Tomossi.