I don't believe there is any inherent damage done by the person who consented to being sexually involved with a married person. The damage done was caused by the cheater, and pursuing another person to sleep with was the immoral action. It's a different school of thought where the consequences (whether you were successful in having an affair or not) don't define the morality but rather the attempt. For example, texting and driving where you almost cause a fatal accident is just as wrong (negligent) as texting and driving where you do cause a fatal accident... whether or not the consequences came to fruition doesn't change that. The cheater committed the immoral action, the mistress is irrelevant. I would leave a boyfriend who attempted to cheat regardless of whether he was successful in that or not, it would not be her fault that he pursued that. Most people believe the consequences have leverage on the morality​ of the action, but personally I view justice as being more blind and objective like I described. Both perspectives are valid.