This is the best answer IMO. Diplomats are inherently there to do, well, diplomatic work, which makes them very vulnerable to political currents. In old times, a king could kill an enemy messenger demanding surrender, as an act of defiance. Other nations could then use it as a pretext to take no prisoners (the Mongols were notorious for brutally punishing any country that harmed its diplomats).
You want diplomats to be able to do their work without political interference, including personally threatening the diplomat or their family. The only way we found so far is to give them complete and unconditional immunity from prosecution by the host country, without the diplomat's home country's permission, because anything short of that can, and is, abused or circumvented.
One important thing is that diplomatic immunity only prevents the host nation from charging the diplomat, and even that unless waived by the diplomat's home country. Now, waiving usually isn't allowed to save face (the home country doesn't want to lift the immunity for its diplomats, even for bad deeds, because it sets a bad "precedent" in their eyes). But, the home country is free to prosecute the diplomat themselves, if the committed action was considered a serious crime in both countries.
In the modern world, there are some prominent recent examples of both how diplomatic immunity is abused, and how it's lack is being abused:
Last year, a US diplomat killed a UK woman in a hit-and-run accident. She promptly fled and returned to the US. She probably didn't even need to flee, because she'd be immune from prosecution anyways, but the flight avoided some bad press before she was back in the US. The UK asked for her immunity lifted, the US refused. So, even close allies don't want to give it up for one another. No word if the US will ever charge her, because under most US laws, unintentional vehicular homicide is treated lighter compared to other homicide, whereas other countries make less of a distinction. If it was straight-up murder, she would likely be charged by the US itself.
Two years ago, the US asked Canada to extradite the daughter Huawei executive and major Chinese political player on white collar crime charges, which many suspected were a pretext in the US-China trade war under Trump. She was in Canada as a private visitor and had no diplomatic immunity. Canada placed her under house arrest pending the extradition hearing. China promptly retaliated by arresting and imprisoning two Canadians on spying charges. They were in prison for two years, until the US agreed to drop charges, and China released the Canadians the same day, making it obvious that the entire affair was a tit-for-tat hostage taking. But the message was sent, and it worked.
So, #1 is a reason of why diplomatic immunity can be bad, but #2 shows why it's an unfortunate political reality, and not having it for people who are political players. Not having immunity means a citizen can be charged for political reasons, and the host nation can then retaliate with similar political charges, hurting everyone involved. If she had immunity, they'd just quietly kick her out and the whole affair be forgotten. And if you say, "well, she'd get away with a crime", well, she still did, except hostages were involved.