A few houses run off in rebellion

The Houses that would rebel are some of the most wealthy and powerful Houses. Anyone who wished to hold Andor without things breaking apart and going to hell would need the support of them. In both Tear and Cairhien, the most powerful and wealthy High Lords/House nobles support Rand even if they plot behind his back. By doing so, they offer some stability to his rule. Those in rebellion are powerful, but extremely diminished in influence. (Riatin and some of Damodred in Cairhien, Darlin and Estanda/Tedosian in Tear) In many cases their loss was either a clear minority or balanced by those who professed loyalty.

In contrast from the Houses of Andor, Taravan would take 6 with them, at least 3 would attempt to stand on their own if not capture one of the aformentioned 6, one would likely always stand aside...to stand against House Trakand and maybe whatever she could kidnap on her own or convince that she's really telling the truth and doesn't intend to turn Andor into a vassal state.

That's not really just "Elayne's perspective". It's echoed by House Taravin (Dyelin), Traemane, those that confronted Rand, Rand himself, and so on.