Yeah that is very odd. Next time if he texted or called to let you know he’s there super early just say “sorry, I’m not available yet but look forward to seeing everyone at x time (when you told them to come over.” Not sure what his intentions were, but no matter what it would be rude because if someone is hosting you’d assume they are setting up, getting food ready, getting themselves ready, etc. I don’t think you have to say something, but if you expect you could be in the situation again I’d text him something like “Next time I’m hosting please come over at or after the start time. You’re coming over early didn’t allow me time to get things ready.” Keep it simple and to the point. Weird!