I realize it's pedantic, but "polygamy" refers to a man having multiple wives. Colloquially, we use it as a synonym for polyamory (which I infer to be the OP's meaning), but the fact that the word is rooted in the idea of one man with multiple women is somewhat relevant to the context of the question. There are few ethical or legal principles that, in the abstract, make polyamorous marriage possible (other than the fact that in most of the 'Western' world we simply haven't done it). However, there are a lot of cultural reasons why legalizing 'polyamory' would quite quickly become polygamy in the pedantic sense.

As long as women remain relatively economically disadvantaged, there will be significant and disproportionate pressure on women to enter into multi-partner marriages that aren't to their overall benefit except in a short-term way. There are cultural pressures that make women relatively disadvantaged as well (and no, I am not making the claim that every woman is at a disadvantage in every way to every man - put the trilby away). A combination of these economic and societal factors will, in all likelihood, result in multi-partner marriages that make women de facto subservients. It is perhaps worth noting, anecdotally, that there are a lot more examples of cults and fringe movements in North America where the (male) leader has multiple (female) 'wives' than the inverse - this, I'd argue, is an extreme example of gendered power dynamics that exist in the population at large.

That is not to say that this isn't something that happens already in single-partner marriages. Indeed, the institution of marriage itself has deeply misogynistic roots. Women often suffer horribly under single male husbands, but cannot leave for economic or social reasons (with the same caveats as before, trilby patrol). Historical attempts to 'fix' marriage have resulted in a set of rules that at various times hurt undeserving women and undeserving men (custody being an obvious example). I can easily imagine that an attempt to codify polyamorous marriages will have a similar track record of really hairy 'bugs' to work out.

Suffice it to say that while there may not necessarily be a good reason not to legalize poly marriage, there are pragmatic reasons.