Why is cultural appropriation bad? I am not a scholar but I have thought of this. Let me start with an example.
The video for the song I Can't Lose by our dear Jewish boy Mark Ronson is set in an Asian-themed location, including Chinese characters. I have a cluster of dear friends who are Chinese, so I asked them to translate. They did so but two, including the most soft spoken of the bunch, were offended by the video.
It wasn't that the video was offensive or mocking. However, it reduced Chinese culture, THEIR culture, to a way of garnishing or exoticising an otherwise average city scene and bar fight. It makes them seem weird, different, exotic, all of which carry the association with being unwelcome and other. My Chinese friends aren't something that should be used to make your life more interesting just because they are Chinese. One is a body builder. That's the interesting part.
Appropriation is also problematic when the appropriators historically oppressed the target culture. It becomes a question of, "What, enslaving our bodies and taking our land wasn't enough? Now you want to take our essence and turn it into your ornament?" When a style is taken from a source and then reinvented by another group, then sold as original and/or better than the source, that is appropriation. (Also, plagiarism.)
The most problematic cases of appropriation occur when religious elements become trendy. Witness all the non-Hindu women wearing bindis and calling them bindis.
It's not just whites stealing (x), by the way. The odd appropriation dynamic between rap culture and Asian cultures is something I wish I knew more about.