Many of them don't take enough responsibility for their own well-being even in areas with good bike infrastructure.
That said, the thing I hate the most about living in one of the most bike-friendly cities is the asshole drivers who don't see the cyclists and then get mad at me for accommodating them.
Like I've got a green light and I'm turning right and I see a cyclist flying up on my right in the bike lane going straight (the crosswalk is green for them), and I know if I turn immediately, at minimum I'm going to scare the shit out of them and cause them to slam on their brakes, so I just chill for an extra second. Without fail every time there is some asshole with a 3-inch dick in his raised pickup behind me laying on the horn wanting me to just kill the cyclist like a true patriot.
I would much rather get rid of all the pickup drivers than the cyclists.