I actually agree with you but I experienced it once in my life during last fifa world championship. That was the one and only time during a game that I really genuinely was 100% into an outcome of any sport, with all the appropriate mood swings as the game progressed and elation at the end. But I think all the reasons were really just a manifestation of personal stakes with the game serving as a metaphore.

But to put it more plainly, if you have animosity towards a group and you know the outcome of the game will piss them off to no end, wouldn't you then take pleasure in someone "yours" defeating them (even if they are only yours in your opponents heads)?