I think Kansas just plays a style of basketball that isn't conducive to winning in a tournament environment.
In the NCAA Tournament, you have to be extremely flexible. You have to be able to push the ball in transition against teams that don't play well at a fast pace, and you have to be able to slow the ball down and execute in your half-court sets against teams that don't play well at a slow pace. Bill Self employs a very deliberate half-court offense that Kansas very, very rarely goes away from. On one night, against one team, this can result in a blowout win for Kansas; on the next night, against another team, this can be a recipe for disaster.
More detrimental, I think, is Kansas' style of defense. They play extremely aggressive perimeter defense and run shooters off the three-point line. This is a fine strategy against a team that relies on the three-ball or has multiple snipers. But against a team that has (i) multiple players with plus quickness and court-vision, or (ii) a player who can create out of the low-post, that strategy is going to put you [in the case of (i)] routinely out of position, or [in the case of (ii)] defenseless against interior movement.
To put it more concisely, I think Self's game is far too rigid to succeed consistently in the Tournament. Good Tournament teams are extremely flexible and spend the entire season preparing for March (intentionally or not) by playing against opponents with drastically different floor preferences and proficiencies. Self teams fall apart when they're forced to play out of their comfort zone.
Side point: I don't think this is a critique of Self so much as a description of his coaching style and explanation of his struggles in the Tournament. For the most part, Self does what works. That's why you can expect his teams to hit 30-wins almost annually. But in preparation for Survive-and-Advance season, sometimes it pays off to do what doesn't work, figure out why it doesn't work, and find a solution in the case that you need it to work sometime down the road.