How does Goedel's Incompleteness Theorem imply that infinity is a poor concept? His theorem simply states that we can never prove everything that we want to prove. Any system that you propose will be subject to the incompleteness theorem.

You've already expressed that you believe mathematics should reflect the universe that we live in. Then wouldn't imposing an arbitrary rule be counter to this goal? If our goal is to discover the mathematics of our universe, then we should only adopt the axioms necessary to make mathematics conform to the way our universe works. This means that until the universe gives us a reason to believe that infinity shouldn't exist, then we shouldn't make up a rule saying, "Infinity doesn't exist." Any restrictions on the number simply reflects the limitations of human understanding at the time rather than the true nature of the universe.

Why do you believe that the concept of infinity is poorly defined? It's difficult for humans to grasp, perhaps, but that's a limitation with humans rather than with the system of mathematics we've discovered. Infinity is a fundamental component of our universe and its mathematics. Otherwise simple things like calculus and statistics wouldn't be possible because their foundations rely on the use of limits. Fundamental forces wouldn't work properly because you wouldn't be able to have asymptotes. How can black holes exist under your model? Ratios and scaling will also break your system. Simple scaling of dimensions will break your system.

While you might be able to fudge your system to be "close enough" to the real outcomes we currently achieve with real mathematics, you risk at some point arriving at a problem where your arbitrary rule holds us back. And if you're fudging your system to achieve similar results to real mathematics, then by attempting to do so you're acknowledging its inherent inferiority to a system without this arbitrary rule.

Most fundamental of all, though, is that your arbitrary rule will make your system worse off under Goedel's Incompleteness Theorem. That is, as a result of the arbitrary rule, your system will have more theories that are true but cannot be proven than the current system of mathematics. If your goal is to understand the universe, then this is a fatal handicap.