This is true. We can't ever know what would have happened. I agree with your assertion that it is speculative. Still I think the question is not so difficult as you imagine.

I think a decent argument can be made that as a whole the importation of slaves from Africa was likely unnecessary and possibly even counterproductive. If we go back to Bacon's rebellion, which I argue was the catalyst for widespread use of slavery in the English American Colonies. We see plenty of labor available in various forms of indentured servitude or sharecropping style schemes. While many of these could be argued to be slavery they were not nearly so bad as the "chattel slavery" African slaves were subjected to.

Once we move to the American Revolution we see a majority of slaves in the colonies supporting Britain against the Patriots because of the British offer of freedom. If that segment of the population were disenfranchised Irish or English immigrants it can be argued they could be a significant boost to the Patriot cause.

Could cotton have been produced as efficiently without a slave work force? By 1870 The US was producing more cotton than had been produced in 1860. Although the US didn't regain enough contracts to export greater amounts of cotton till 1880. I think this quite convincingly argues that a non slave labor force could equal if not exceed the slave labor force.

I have thus far failed to find any evidence that supports the importation of Africans for "chattel slavery" as a necessary evil. Other sources of manpower were available. Off the top of my head workers could have come from Ireland, England, Germany (where many German princes would have happily shipped a considerable number of unemployed mercenaries to overseas colonies to reduce the problems they caused at home) or perhaps even the Barbary states where European slaves could be ransomed and then brought to The United States to work off the ransom debt.

Perhaps this would merely have exchanged African slaves for European slaves. Still the defining factor as shown by Bacon's rebellion is that European slaves because they were not defined by skin color could much more easily be integrated into the general population. African slaves and their descendants were easily identified by skin color for discriminatory purposes. This discrimination hindered integration of the former slave population with the general population and led to many of the problems associated with "black people" that we see today.