Your conclusion that it's not a fallacy seems correct, but just not your reasoning why.

It is true that if one wasn’t invited to the party, they wouldn’t have a hangover.

At first glance it seems a little trickier than that. There's two separate facts connected by a dubious chain of causes.

1. He sent me an invitation to that party, and ⋮
2. I have a hangover today.

What you're claiming seems to be that if the invitation hadn't been sent, then they would not be hungover. That seems plausible, like if some link in a chain of events arriving at the presents is broken, then the present will (or even must) change. However this assumes that an invitation was (a) necessary (cause) for a hangover. This is in ordinary informal senses obviously false.

Someone might go to a bar instead, or drink at home, and get drunk and be hungover the next day as easily as go to a party, get drunk there and suffer similarly afterward. An invitation is clearly not necessary, but only sufficient under some pretty limited circumstances, like say for a person who only drinks socially and on special occasions. Generally though, there's no strict logical implication that if an invitation is sent out a hangover will ensue.

Formally its I→H ¬I /?

Can we conclude ¬H?

Answer

Entertaining ideas of what might have happened if links in a causal chain differ or were broke, like if an invitation had not have been sent, after the fact, is quite natural and are called counterfactuals.

If we knew the person was an alcoholic and they always serve alcohol at the party, then you get build a case for an inductive argument that yes without the invitation there would not be a hangover. Inductive arguments require much for evidence, or further facts. As to OPs turtle contraption informally it's also pretty clear and distinct. A turtle put in such a precarious position is effectively doomed to fall victim to the boiling water. Whereas, ceteris paribus, for most people, merely receiving an invitation to a party they can easily avoid a hangover the next day, merely by exercing some very plausible choices.

So no fallacy, though you might justly complain of them making a silly clamour over a tortured chain of events.