Here's what you should do: go to the character sheet of each of your vassals that hate your character and hover your cursor over the box that shows heir opinion of you. In the tooltip window that comes up, you can see every single modifier that affects their opinion of you. What you'll probably find is that it's just a long list of small things that make them hate you. Things like:

Envious -15 (caused by the envius trait) Foreigner -20 (this vassal has a different culture) Proud versus humble -10 (characters with opposite traits dislike each other) Attraction to gluttonous -10 (heterosexual characters of the opposite sex have an attraction opinion of you based on your traits)

Just this very short list of opinion modifiers adds up to a total -55 opinion, and my point of showing you this is that what you'll often find is that characters who hate you just have a very long list of small negative modifiers.

Your counter to this is diplomacy and proper education. Once your realm grows so large that your vassals combined are more powerful than you, your single most important stat is diplomacy, because vassal opinion is the most important factor in realm stability, vassal levy size and even taxation. So what you need to do is educate your heir by grey eminence and aim for traits that boost both diplomacy and vassal opinion, such as just, brave, gregarious, kind, ambitious and even trusting if you can afford it.

The child of a concubine trait means nothing. All it does is give your character -1 diplomacy. There are no special opinion modifiers, no special events, no religious opinion of it, nothing. It can't even be inherited. The -1 diplomacy doesn't mean much when you have some 20-30ish diplomacy anyway.