No, he can't forgive himself. But he can't blame himself at all. That's his failing. He can't bring himself to say that he did wrong by his own moral code. So he can't even admit that he can't forgive himself because he can't admit that he did the wrong thing.

He clearly saw what he was doing was wrong. He saw the things he had ordered, after he had that talk with his sister, and regretted his actions. He told his men to stop doing what he had ordered them to do and leave the survivors to rebuild. He had returned to purge them because of their rebellion his way. The Emperor could most definitely forgive him for that. He was broken finally when he realized that what he had done in response was the wrong thing.

So he made up a lie, to comfort himself. He said that the Emperor couldn't forgive him. That changes everything. If it's the Emperor who can't forgive him, then it meant that Perturabo hadn't done something necessarily wrong, just that he had done something that went against the Emperor's morality or code. And thus by turning on the Emperor, by calling Him false, by supporting Horus in the idea that the Emperor Himself is wrong, Perturabo has done nothing wrong. He didn't fail.

The Emperor failed.

At least, that's what I got out of it.