AN 5.161:

"There are these five ways of subduing hatred (āghāta) by which, when hatred arises in a monk, he should wipe it out completely. Which five?

"When one gives birth to hatred for an individual, one should develop good will for that individual. Thus the hatred for that individual should be subdued.

"When one gives birth to hatred for an individual, one should develop compassion for that individual. Thus the hatred for that individual should be subdued.

"When one gives birth to hatred for an individual, one should develop equanimity toward that individual. Thus the hatred for that individual should be subdued.

"When one gives birth to hatred for an individual, one should pay him no mind & pay him no attention. Thus the hatred for that individual should be subdued.

"When one gives birth to hatred for an individual, one should direct one's thoughts to the fact of his being the product of his actions: 'This venerable one is the doer of his actions, heir to his actions, born of his actions, related by his actions, and has his actions as his arbitrator. Whatever action he does, for good or for evil, to that will he fall heir.' Thus the hatred for that individual should be subdued.

"These are five ways of subduing hatred by which, when hatred arises in a monk, he should wipe it out completely."

TL;DR: Sans mudita, lest one become a psychopath or something. Of note in the case of humor is that the Visuddhimagga designates 'merriment' as a near-enemy of mudita. Furthermore, the subsequent sutta at AN 5.162 goes into quite some detail on an individual basis.

Fear, on the other hand, is a bit different. The Bhayabherava Sutta (MN 4) has more on this, as does others I'm sure.