It's important to recognize that we can have multiple group identities without necessarily wanting to segregate ourselves from the rest of society. Let's say, for example, that I identify as a member of "Red Sox Nation." It's pretty clear what I mean by this, in part because the baseball team has strong trademark laws in its favor. Nonetheless, I know folks who get really annoyed at all the people who started wearing Sox hats but don't really follow the team. They feel like this devalues what it means to be a Red Sox fan, which is a meaningful part of their identity.

Concern about cultural appropriation is a bit like this. Except that we're talking about minority groups which have historically had the power to control how their identities are perceived taken away from them. Take the Native American headdress, for example. This is something which carries a lot of significance in Native American culture. But if other Americans start wearing it as a novelty fashion item or prop, then that's what it becomes, because they have a lot more power over the broader culture than Native Americans. And since Native Americans aren't segregated from the rest of society and live within the same broader culture as the rest of us, this takes away from their power to define what it means to be a Native American. When people talk about cultural appropriation, what they're trying to do is raise awareness about this power imbalance.

The thing with hip-hop is similar. Some people feel that the history and cultural significance of the art form are important, and get upset at people whom they perceive to be adopting the aesthetic elements and discarding everything else. Some people also suspect that Iggy Azalea is deliberately trying "sound black" for entertainment value, which is reminiscent of racist minstrel shows.

Of course, it's possible to be oversensitive about these things, and reasonable people can disagree.