You said you were annoyed by people who read books on body language. The context of the comment led me to assume that you are annoyed because you believe the study or belief in body language is dumb.

I can't see why else you'd be annoyed, other than you think poorly of books on body language. Certainly, you wouldn't be annoyed if you thought it was a good thing to read books on body language.

So, proceeding from the rationale that you believe books on body language are dumb (and are correspondingly annoyed when people read them), the cause of your dislike for books on body language must be because you don't like the topic of body language. I surmise this from the context of you agreeing with the previous comments, above, which support the belief that interpreting body language cues is like reading a horoscope or other junk science.

The alternative may be possible, that you are annoyed by people reading books on body language because you are annoyed merely when people read books at all; but while that may be logically inferred, there isn't enough evidence to support that finding, especially given the context of the previous comments.

So, you are annoyed by people who read books on body language, not because you're annoyed when people read books, but specifically because they're reading books on body language. It annoys you, not because you hold the science of interpreting body language in high esteem, but rather because you think interpreting body language is stupid and pointless; and subsequently, reading about interpreting body language is a stupid and pointless waste of time, which annoys you when people do that.

Now, since body language is nonverbal communication and because nonverbal communication is inherently "body language" (I'm struggling to think of examples of a person physically communicating in a nonverbal manner that wouldn't be considered "body language") it stands to reason that your outlook on body language is actually your outlook on nonverbal communication—because both are one and the same.

Thus, you are annoyed when people read books on interpreting body language, which functionally are books on interpreting nonverbal communication, which is a subject you hold as junk science or a waste of time; and so thus does people reading about nonverbal communication annoy you.

So I ask, you don't believe in nonverbal communication?