Several reasons, most of them not for good reasons.

1. Ageism. Most kpop fans are teenagers and young adults, JYP is much older than the idols they follow and seem its weird that he's injecting himself so much into the lives of the idols they follow.
2. A lot of people don't like JYPE's decisions and they assume that JYP has complete control over the company JYPE, when he really doesn't. There's a board, executives, and upper management that handle a lot of decisions within the company. However fans don't know who those people are because they don't make themselves publicly known. JYP is a public figure so he's easy to use as a scapegoat.
3. He has made some questionably insensitive comments and actions in the past.