You have to trace it back to why they exist. Pre 90’s very, very few diesel cars, almost exclusively used by taxi and traveling salesmen types where it was all about getting the lowest cost mile.

Then emissions really started to come in and they only cared about CO2 emissions at first. Nobody was paying much attention to particle and nox emissions which are far worse to human health, so you got to market diesels as efficient and clean. Add on a turbo and you have good emissions, good economy and performance - basically a whole generation of car buyers were suckered in. Then emissions regs started closing the net, and frankly, no turbo diesels aren’t clean enough for the city, but automakers, particularly VW had built such a market on them they just kept trying to push forward rather than emit defeat. And that’s basically it - once you start focusing on particle and nox emissions, high performance diesels aren’t favourable.

The worst thing is so many people knew this for so long. When I was a kid in the 80’s we did this thing in school where you went out to a tree by the road and rubbed tissue paper on its leaves and got all the soot, and we were told that was diesel spot from all the trucks and busses (okay it wasn’t all that, just pointing out, everyone knew they were dirty long before). I remember bringing this up in the mid 2000’s on a forum much like /r/cars and just getting shouted down to hell - modern regent systems get rid of all that, turbo diesels are greener and such, but I also kept reading from people who knew more about it that yes, they’re bad for human health but they’re better for greenhouse gasses and that wha people care about.