NTA It IS the mom's fault. However, at this point you are punishing the kid for his mom's shitty behavior. Could you consider dropping off James yourself once everyone else has been picked up if his mom is over an hour late next time? I know it's not required if you and could be seen as enabling his mother's behavior, but it is sad that James has to miss out on seeing his friends when he probably has enough to deal with with a mother like that. That or you could consider having James over separately and always arranging that you'll drop him off

Obviously you're not obligated to do anything though. It's just a suggestion since James himself has no fault in the matter, but he ends up being the one most affected by this.