I definitely think you're on to something, though I'd argue that the patriarchy does exist, but as a byproduct of evolution and reproduction. The very fact that men are on average larger and stronger than women already creates a power imbalance - why connect with a woman when you can stick around to raise kids when you can overpower her, get her pregnant, and then nope out?

Many animals do the same, some even to the extent that their reproductive physiology evolved to protect from rape. Pregnancy is a huge risk with many many possible complications, and it leaves one basically incapacitated for at least a few months.

It's not a stretch to say that the consequences of our evolution and reproduction are mirrored in our society in a way in which men hold the power. Because of the strength imbalance, historically men were the hunters and protectors, and as a result, physical strength held great value. Men held the leadership roles within their tribes and villages, and were the ones to make decisions.

Weakness and cowardice were likely shamed in primitive societies, and as women grew to be associated with those qualities due to their stature and the risk of pregnancy, being a woman would likely also be seen as shameful. The power structure and values of society grew to reflect this.

Our society today is simply an updated version of these primitive structures. And despite some of the best intentions (though they aren't always good), the men who hold power often don't understand the effect their decisions have on women, because they never have to live it.