Interestingly, this is a narrative I have also parroted in the past, but FBI crime stats for 2017 (https://ucr.fbi.gov/crime-in-the-u.s/2017/crime-in-the-u.s.-2017/tables/table-43) show that whites and blacks were arrested for drug abuse violations at a rate that very closely aligns with the proportion of the population. (To be fair, these stats seem to be for arrests, not convictions, and sentencing disparity is another issue you brought up).
Do you have stats on crack use by race? Just wondering where you're getting "crack cocaine..generally used more by African Americans." The disparity between crack and cocaine sentencing does seem like it would disproportionately affect the poor, and Blacks are more likely to be poor.
I absolutely don't think non-violent drug possession warrants jail time. Where are you getting the claim that the majority of people in prison are there for drug offenses? Looking at the FBI stats, it looks like the highest single reason for arrest is drug offenses, but the combined number of people arrested for assault and theft, to name a few, is higher than arrests for drugs.
A huge problem is that a large number of people sitting in jail have not even been convicted of a crime, but are awaiting trial and cannot afford bail. This is going to disproportionately affect the poor, and again, African Americans are more likely to be poor. This is really an awful situation that needs to be fixed.