Who was the physician who "diagnosed" with the IBS label? Is he your primary care doctor? A specialist?
I don't know your situation or your familiarity/trust in your primary care doc, but when I had gut problems I moved on from an unhelpful primary care physician to one that would take my symptoms, problems and concerns seriously while working with me.
I found help in the form of a primary care doc who incorporates a functional medicine approach to her practice. You can search for similarly qualified individuals based on your location and needs. That isn't a prerequisite. I'm sure there are poor functional medicine MDs the same way there are poor internal medicine MDs, unhelpful specialists or anything else. This is just what I found in my own search for relief and it's what helped me.
With your symptoms I would absolutely seek out a gastroenterologist, specialist on the digestive system. They can work with you and rule out the serious, easily diagnosed issues (Crohn's, Ulcerative Colitis, Irritable Bowel Disease, Infection, Parasite, colon cancer).
When approaching doctors just come armed with knowledge but come armed with questions too. You're not a professional in the medical field (I'm assuming) and they are. If you come in with questions (is what I found online bullshit? Why would I have BLANK, or why would I not? What tests can we run to rule out X, Y & Z?) then an good doctor should be able to answer them. A good doc will steer you in the right direction, work hand in hand with you, seek to give you better quality of life than you're currently getting with your symptoms. Only you can draw the line and interpret whether any doctor you're seeing is helping you or worth going back to. When I was at my worst I had a lot of new patient visits and went through a handful of doctors before I found one who essentially saved my life. I wasn't diagnosed with a fatal disease, but I had no quality of life.
If your symptoms are having serious impact on your quality of life then stress that to your doctor, make sure they know. If you're in daily pain, experiencing chronic stress due to these issues then don't be shrugged off with the "IBS" label. It's not normal to have random, inconsistent stools. It's not normal to have daily abdominal pain.
With your diet being so restricted be sure that you are getting as much nutritional balance as you can. I lost a lot of weight when I shrank my diet and had to begin monitoring my protein, fat & overall calorie intake to stay healthy. Supplements helped me greatly when getting back on track, to both ease symptoms and keep overall balance in my nutritional intake. You may not need to dive as deep as I did but I took 1000MG Fish Oil, 1000MG Vitamin C, Vitamin D, N-Acetyl Cysteine(NAC) & a daily VSL#3 high strength probiotic daily (and still do for all of the above). I also had 4 recurrent bouts of C Difficile infection though. Metagenics, OrthoMolecular Products, Walgreens & Finest Nutrition were brands I used but as long as you're not getting low dose stuff or anything with unnecessary additives it's not a big deal (most of those available online, I'm in the US so not sure about Canada). GlutaShield is a powerful product from OrthoMolecular as well.
Dr. Mark Pimentel, director of both the Gastro Motility Lab & Gastro Motility Program at Cedars-Sinai, has tons of great research in the field of IBS and IBS treatment (if you end up back at that diagnosis once you've ruled out other biggies). He could be a great resource.
I had to have my gallbladder removed but years went by before any doctor cared enough to have it checked. If you don't find a diagnostic answer to your abdominal pain then I would suggest ruling the gallbladder problems out, even if a scan doesn't show stones (I didn't have stones but a HIDA scan showed my gallbladder functioning at under 15%). Depending on how long you experience symptoms (and how severe they get) other specialists may help you paint a clearer picture. You can always ask a helpful Dr for advice or referrals and they should be able to assist you.
My case was very severe, very hard to diagnose, and it took a very long time. I jumped through a lot of hoops and had a lot of headache trying to find help. Your situation sounds less daunting and I hope 90% of this ends up being irrelevant and you don't need it. But if you do, I'd like to give you any insight I can (or anyone else with issues reading this sub).