I used to reply with a polite email that just said something along the lines of "Thank you for your interest, but we have decided to pursue other candidates at this time." It's always led to a snide email or two, but nothing serious.

About 5 years ago, for the same position I'm advertising for now (the guy who filled it previously is getting promoted out), I ended up getting about 9 applicants who were hostile about the fact that I rejected their application "for no reason" but still had the job post up. Most of them were content with sending me hate mail daily. Two of them kept calling the office to tell me (and later when I stopped taking their calls, the receptionist) what a piece of shit I was for not giving them a chance. And one of the two came down to the office and threw a fit in front of a bunch of clients. When I had them removed they found a phone number online they thought was my home number (but was actually my wife's cell phone number), and made a few threatening calls to her.

This shit is the reason all the other department heads just don't respond to applications they're not interested in, or post ridiculous standards to apply, or has HR do their automated "keyword hunt an online application" thing. It's also why I stopped responding to applicants I wasn't interested in until the job was filled.

But I'm the crazy bastard who still insists on personally reviewing and responding to every applicant, so the burden of handling the consequences of running my department that way falls on me. It's just a lot more complicated than "so what if you hurt someone's feelings?"