Yeah, there is several counters to smart riders. Easiest one is just attacking the horse with ranged weapons. It's a big, easy target, and even better, there's no way for it to get HP back. It's easier to hit and to kill than the rider himself. Shooting at the rider accomplishes nothing, he can just retreat and heal back up. That's the biggest mistake people do. Shooting at his horse does permanent damage to it.

Second is toolbox, because while the best horseman can keep spacing away from billhooks and other dangerous weapons, even the best horseman can do very little about someone literally spawning an objective directly in his path that causes his horse to buck.