I personally feel that the visitation rules should apply equally to both sides of the family. So if one set of grandparents get access by a certain day after birth so should the other set of grandparents. Same with aunts/uncles of the baby - if my brother is coming to see the baby 2 days after birth I feel I should let my husband’s siblings come within same timeframe.

I’d just set expectations and boundaries - “I’m not sure how I’ll be feeling. We’d love you to come but I think I’ll only be up for a half hour visit. If you could call beforehand we’ll let you know when the best time would be to hopefully get a time when baby (and the rest of us) are fairly well rested and happy” something like that