"Back in the day, people had lots of kids, no family and were dirt poor and the kids survived anyway."

The opposite is true. The nuclear family living alone without help / contact with extended family (grandparents, siblings) is something which is completely new. Kids moving out as soon as they finish high school to pursue a higher degree is something new.

That said - if you want to take care of kids, get kids of your own. It sounds like you and your mother want to inject yourself into the family.

Why the fuck are you part of deciding the babies' names? ("when we were talking about baby names" WE! Why the fuck WE?)

Why the fuck do you want to be there when the kids are born. Mother & father is the usual setup.

When your mother wants to visit her grandchildren, as simply "No, thank you" would've been enough. But it would surely make your mother "feel crushed" again.

You offered to take care of the children. It was politely declined. Nothing wrong with that.

How the fuck do you know that they hide the kids, when the neighbours are around to see them (wtf neighbours asking to see kids??)? It surely shows that the neighbours and you have a close enough connection.

Now, I get that some people like their space.

You finally got the right clue.

Your mother is inviting over herself. You are offering to take care of the kids unasked. The neighbours are forwarding information about the kids to you / your mother. They cannot parent alone. Your family is constantly interjecting.

Talk to your brother. Tell him you'd like to see the kids more, if there's an option to do that. Every week / every fortnight / once a month.

If he declines, ask him to tell you the honest reason why that is. If you are pushing too much. Don't be angry if he tells you yes.