It really comes down to a timing and the gradual nature of agricultural-development as well as perceived risk, depending on the strength of the effects.

You could argue sugar was good for you before they knew how bad it is for you, because it made you feel good, it tasted good, and there were no immediately obvious long-term effects. Considering the conspiracy to cover up the science showing the health effects, it's hard to blame people.

I think the proposition that "substances get banned because, they are dangerous, addictive, and destroy lives" is a red herring. If that were really the governments aim, what are they doing about opioid manufacturers?

Substances are banned because banning that substance serves someone's interest. It might be political, personal, religious, business-related, or some fun combination of the above.