If it were merely a hormonal issue, then trans men would dominate (or at least hold their own) in male sports, which absolutely isn't the case. Where are all the high calibre trans male athletes? Male frames are much larger and have higher bone density than females. It's like comparing a cis male human athlete to a gorilla.

Regarding your comment about trans women who start HRT early in life - that isn't what's happening, especially with athletes. Every trans athlete I've seen appears to have just begun transitioning within a few years of entering female competitions. Starting estrogen treatment in your mid-twenties and then competing in the olympics is not even close to the scenario you're describing.

This is a bad situation since it precludes the possibility of anything like a fair competition between trans athletes and women. In the case of contact sports, there's a real possibility of severe injury or death for the cis females, which is exactly why most sports are not co-ed.