YTA. You cheerfully invite his family to spend time with the children in a way that you totally control. The first time you cheerfully get invited to spend time with the kids that you don't control at all you get quite upset.

See the problem? You like the control, you enjoy it. Get used to not having it as time moves forward. If you don't reign yourself in the kids will eventually figure out it's gross behaviour and spend far less time with you as they age.

The reason they don't come over to your house or these things you invite them to is because they don't like you, how could they? You want total control over your children's lives and any access to the children by your ex's family or by his.