I remember in high school I got called racist by other kids and a teacher because I said I didn't think affirmative action was fair. I also included gender in that; as a woman, I would feel incredibly uncomfortable knowing I got a job over an equally or more qualified man just because I have an innie and not an outie and the company needed more women to look more diverse.

Because all that does is tell me that a company cares more about how it looks than it does about getting a job done right, which gives me a negative impression of its culture. If I earned the position, great. If I didn't, hire the other person and let me find a company where I am perfect for what they need.