This might hold up if people always used "very" and "really" for the same function that they (incorrectly) use "literally". You don't hear people saying: "Really, I can't do this." , unless they truly can't do it. If some did use this sentence it in a context where they /actually could/ do what they said they "really" couldn't do, I would challenge them, just like I would challenge them (and mock them) if they used "literally".

Use as a true intensifier:

Using "literally" as a true intensifier doesn't bother me, because it just amplifies a statement that is already true.

"I am literally mad right now."

No one uses it like this, but this would be using literally as a true intensifier, the same way you would use "very" or "really".

Let's examine an annoying usage of literally. Suppose a woman who's been standing in line at Starbucks for 1 minute says: "This is literally the longest line I've ever waited in." I think it's fair to assume this is not true, and thus an incorrect usage of the term "literally". If the woman said: "This is really the longest line I've ever waited in.", I would be annoyed for the same reason I would be annoyed if she had used "literally". Also, it's hard to use "very" in this improper context, unless you use the longer form: "verily". I don't think I could ever be mad at someone for using "verily" in this context, because they are almost certainly using it in jest. The only time I would find the misuse of "literally", "really", or "very" (verily) acceptable is when it is obviously used in jest. I think it can add humor when it is misused intentionally.

Now, let's go back to the comment that caused me to post my The Oatmeal link:

Fuck that meme. Literally.

If he had said: "Fuck that meme. Really.", I would have said basically the same thing: "Really? You actually want to fuck that meme?" This misuse of "really" is obviously to most people, thus I wouldn't need to explain myself. However, when it comes to "literally", most people don't understand its real meaning, thus the reason I posted the The Oatmeal link.

The reason why you and others probably think this little quip from Marc Ettlinger makes so much sense is because you are accepting the comparison between the common use of "literally" and the common use of "very" and "really" as intensifiers. This isn't a good comparison, when "very" and "really" are used as intensifiers, they are often used in statements that are actually true:

"I'm really hungry."

"I'm very sick."

Both of these statements are logically consistent, and this is the common use of "really" and "very". However, when "literally" is used, it's often in a statement that's not actually true:

"I am literally freezing."

If this person was literally freezing, they wouldn't be able to speak. When people misuse "literally", I often feel like they aren't doing it for the humorous effect, I believe they just don't understand what "literally" means. I find this annoying.

So, hegemonistic, it's neat that you found this little excerpt, and you probably think you're really clever even though you parroted it without thinking it through, but I am in fact not hypocritical with my views. I do find it annoying when people use "really" and "very" in statements that aren't actually true, it's just that people don't do that nearly as often as they do with "literally".