The higher taxes on non-Muslims leading to conversions actually was viewed as a problem for the early Umayyad state. Syria was its core province economically and politically, and was overwhelmingly non-Muslim. As people began converting to Islam, tax revenue began falling.
The Umayyad solution was to impose additional taxes on Muslim who were originally Christian or Jewish, as well as their descendants, such that conversion could not be used as a mean of lowering an individual's tax burden.
This was highly controversial at the time amongst Muslims, as many argued that there was no legal basis for this. But it stuck, and became one of the legal precedents for secular taxes in an Islamic state.