You're lucky; it sounds like you've rarely if ever had a job where you routinely have 20 patients to yourself, or one where consultant ward rounds often go on until well into the afternoon. There's certainly geographical variation (I've had friends move to the area where I work who've been surprised at how sparse staffing is compared with their previous location), but I've had several jobs where there literally isn't time in the day to squeeze everything in (mostly due to chronic under-staffing or rota gaps and the workload generally being too much for one person), and there are plenty of things it's inappropriate to hand over/would take as much time to hand over as to just do it yourself. I'm not going to ask someone to speak to another specialty for me about a patient they don't know, or break bad news to one of the patients I've been looking after, or put out tomorrow's bloods for me.

Even if it isn't me who has too much work to squeeze in, there's a reasonable chance it'll apply to someone else on the team, and I don't leave work if there's something I can do for a colleague that means they might leave a bit sooner than they would have if left on their own.