About the marks - I go to an IB school in Ontario, and here, our mark conversions give us really high Ontario marks. Anything above a 5, and you're getting at least a 90, which is enough to get you into a lot of Canadian universities. You seem to be putting a lot of effort into school, which is good, because it helps you get a lot out of the program. Even when you feel like you're doing bad in a class, conversions to Ontario marks will save you. For example, I'm in HL math and I estimate my raw mark from in class tests to be in the high 70s, but my Ontario mark in January was a 95 (level 6). The school compensates you for taking such a rigorous program, especially since it's filled with things you might not want to do - the EE, TOK, CAS reflections, courses that you don't want to take but have to etc.
Also, it's good preparation for university because you really learn how to learn on your own since it's so fast paced - I learned most of SL physics on my own (looking back, the material is really easy), and I've learned the past 2 semesters of HL math on my own too cuz my teacher is bad. So the program will prepare you well for university if you put the work into it.
On the other hand, if you really want to get into an elite university, you have to have a lot of extracurriculars with which you can actually convince the universities that you did something significant. Whether you can do that depends on how you evaluate your own time management, so that's up to you. However since U of T will only consider your converted Ontario marks, you shouldn't have a problem getting in (many of my friends and I have gotten in already). I don't know how mark submission works for Harvard and Stanford, so I can't comment on that.
With regards to you sacrificing a lot of time and effort, it gets better. Sure you have less work, but you'll probably get less stressed about it because you'll be so used to it (although this is just my personal experience). It really helps if you like the subjects you're studying, so make sure you won't hate any of the courses you choose. Obviously you can't be passionate about everything you study, but as long as you don't dislike it you'll be fine.