Nice work man. I have a similar story, although I realised when my dad suggested I drop out of school and do a trade (builder, electrician et cetera) after I finished year 10 (high school). I always liked maths and science, though I never applied myself in school. Wasn't in any of the top classes for maths and science and was in the bottom english class.
My dad is a maths teacher and was dux of his school. His 4 brothers and 3 sisters are also maths teachers (they only became teachers because my grandparents couldn't afford to send them to university and it just so happened there was a lack of maths teachers in the state, so they all got scholarships for maths education), the youngest brother is an elec engineer (my grandparents had more money then, as the rest of the kids had left, so they assisted him during his degree). Four of My cousins have done engineering and another law. On the other side of my family my aunty is a doctor and works in Pharmaceuticals and my uncle is a geologist.
So as you can see I felt that I needed to step up my game. I felt like I had natural talent but was really under performing. at the beginning of Year 11 I started studying, continued through year 12. That paid of and I got into engineering at a target university with a scholarship with the largest contractor in my country. Parents couldn't really believe it. I only scraped into the degree and the scholarship seemed to be awarded due to my interpersonal skills I displayed during my interview rather than my grades.
My scholarship required me to keep an equivalent of a 3.0 GPA. I was nervous of loosing it so I stepped up my game even more during my first year of university and became really interested in physics and maths, often learning in my spare time. Though i still felt like was feeling the drag of my lazy younger years, having to work harder than most. I got an equivalent of 3.4 GPA in my first year. I kept this up for a few more years.Fast forward to now, I'm in my 4th year, doing a double degree engineering/commerce (5.5 yr long degree), equivalent to a 3.7-3.8 GPA and doing my honours thesis.
My extended family now takes more interest in me, definitely feel more accepted. My uncles often now talk to me about physics et cetera. I'm still pursuing my interest in maths and physics in my own time and am now leaning away from engineering and more towards IB. Still undecided, but the financial markets seem more intellectually stimulating to me, more so than construction.
Keep at it man. I feel like the more you try the more interesting it gets because you understand at a far deeper level.