Yeah he definitely should have formally broke up with Audrey. It wasn't okay for him not to. I think he realized this, hence the apology in D3.

However as a previous post on this sub said, Audrey attempted to force Ben to marry her in Descendants 3 after she cast the sleeping curse. She threatened him with that she would only lift the curse if he did. And I do think people let Audrey off easier than Ben because of gender double standards, I mean imagine if Audrey were male and Ben were female in that situation. The sympathy would go towards Ben and away from Audrey.

So yeah, what Ben did was bad but I think it's clear that Audrey reacted far far worse. I don't mind people criticizing Ben for that, but I do wish people would keep the same energy for their precious Audrey's worse behavior.