I'm not seeing the logical connection between the previous statement and the conclusion that mathematics isn't fundamental. Only means one thing to me, which is that spacetime in our universe isn't hyperbolic. It could be flat or elliptic, both of which are well-defined mathematical classes.

And on top of that idea, what kind of description is "the universe just is the way it is"? Laziest explanation possible. I can't think of a single logical way for anything to function or interact with other things, or even itself, aside from following a mathematical structure.

The standard is empirical data.

But where does that empirical data come from? Why are there patterns in empirical data? And why is that empirical data the way it is and not some other arbitrary way? These are questions that you are categorically ignoring. And you know... you can use mathematics to create a model from the empirical data, so if anything, that is a hint that there is something more fundamental about mathematics to reality.

When you keep digging deeper into what there is to find in physics, it gets easier and easier to see clear mathematical structures. Eventually, you get to a point where nothing looks like a physical object, and everything looks like a mathematical abstraction. Ultimately, how can you possibly tell if there is a difference between the mathematical structure that defines that particle, and the particle itself? That mathematical structure would look and function in the exact same way as the "real" particle, so I don't think there is a difference. As for why it is the way it is, and why it didn't pick some other path or set of mathematical laws? Well, quantum mechanics seems to be pretty good at explaining part of that. The rest... well, you could try to explain it away by saying there is an unknown force controlling it. But then why does that unknown force work the way it does? The only way to continue logically is to conclude that the universe has the laws it has, because it can. Everything else involves assumptions about the existence of some other kind of force, which is then also unable to be explained. There is no hidden complexity in the mathematics. And this is nothing like asserting that there is an omnipotent intelligence with a creative streak who decided to pop this world into existence. Anything that can happen, will happen.

Instead, mathematics is merely a human language.

There does not need to be two mathematical languages. If we already have a language that we use to communicate mathematical ideas, then what do the mathematical ideas communicate aside from themselves? Mathematics is not a language. Mathematics is a system of relationships between sets and operations, which we use some special language to describe. I would say that if anything, language is a class of mathematical structures.