Some of the decisions were rather arbitrary at best, and racist at worst, though. The sentence disparities regarding, for example, cocaine depending on how you are using it was designed to punish black people more harshly. Opium bans had as much to do with anti-Chinese sentiment than anything.
I'm not arguing that drugs should be legal, but we do have to be clear that the reasons for banning them and the punishment are not necessarily rational.
That’s pure historical revision. The sentencing difference was created in 1986 based on the belief that it was more addictive. It wasn’t until a decade later that research showed the causation had been reversed (more addicted people were more likely to use crack). If you look at the timing, there was a huge increase in drug crime that occurred as a result of the crack epidemic: https://www.nber.org/digest/oct18/lingering-lethal-toll-amer....
The recent change in policy simply reflects the prevailing trend of reducing disparities in sentencing for criminals while increasing disparities in crime victimization by failing to enforce the law.