The fact that you're constantly thinking aboity having sex with other people than your partner makes it seem like you're already waving on your commitment to them. I'm bi and have never been with a woman but I am happily married to my husband and nothing is going to change that. I don't feel any urge to have sex with women the same way I don't feel an urge to have sex with men because I'm in a committed marriage. He's also bi and is on the same boat as me, neither of us have any urge to cheat. If this is becoming a really strong urge definitely talk to your partner about it, if they don't give a free pass to experiment then I think you should consider talking a break or just breaking up so you can experiment and see where your sexuality lies.