A little bit the asshole ; there's a lot going on here.

1) they judge you for a miscarriage: unless you were snorting drugs and drinking, that can hardly be your fault. They are definitely toxic for doing this.I hope your husband defended you, set his parents straight and they apologized and everyone can move on. If not, I understand you never want to see them again.

2) But you mentioned multiple times that he can still see them whenever he wants, he can do for his birthday what he wants (aka spending it with his parents) and he told you he'd also celebrate it with you after. How is he supposed to know you don't want him to do that, if you literally agreed to it and said multiple times he can still see them? Be honest with your husband, let him know what you do want/that you'd be hurt if he goes.

You can't tell him A and then be spiteful cause he did A instead of B. So yes, a little bit the asshole cause you ate all the cake you were going to use to celebrate together, after you gave him permission to go to his parents first.