Family being completely irrational and indulging in clearly self-destructive habits that estrange them from everyone else.. I've experience that firsthand and offer my sympathy. I am sorry that you and your family are suffering through this.

I also have a stepsister from my stepfather, and she went through exactly similar circumstance where her mother is living with a guy who tried to have sex with her (after her mother and the guy were married), from which she escaped. And as I am her drinking buddy when she wants to vent her thoughts and frustrations, I know how much it hurts, the worries, and the feeling of betrayal.

For your situation, I think it is important for you and your family members to actually recognize and talk about the root cause: loneliness and trouble finding purpose in life. As I am young as well, I have no firsthand experience but I have heard that one of the most difficult moments in one's life is after the children have grown up and moved out. Many find it hard to fill the empty hours that ensue from not having the children around anymore, and even more importantly, many struggle of feeling of emptiness and meaninglessness.

Some adapt, some do not. It helps if you have a job that you can focus on, a hobby that you genuinely enjoy, a social purpose that drives you, friends and spouse that make you feel accepted and not lonely. I think it would be safe to say that she felt extreme loneliness and loss of direction.

In this situation, I see only one way that you can try which might improve her situation. You, your family and friends should make clear communication to her that we welcome you and we want you back, but not with him at your side. That he is not welcome in any circumstances. Because he is changing her. That he will abuse her again. Vague hints that the two of them are welcome will make your mother feel that she is not welcome. She must not believe so.

Also, finding something that your mother could enjoy as a hobby, and finding some way that she could satisfy her need for social company and attention is necessary as well. She is returning to him because despite all his faults, at least on some days he makes her feel needed and desired. And she needs to feel needed and desired. If she leaves him, then playing matchmaker might be an option, as people can vet for her since she is clearly not being a good judge of character.

Of course, you must make peace with the fact that all your efforts might not make your mother realize her mistake and change her. It is possible that your mother would tell her new spouse about how the family does not welcome him, which would cause him to poison your mother's ears and try to turn her against your family. That state could go on indefinitely, but at least when your stepfather abuses her as he most certainly will, maybe she will remember what her family said about him and leave him and come to you.

Or, you could also simply forget about this, as your mother is an adult and she has made her choice, and the results are hers to bear. This is certainly cruel and unethical by almost all standards, but know that there is such choice, weigh how you feel about each, think on how much you can bear, and make your decision. Writing down your thoughts in a journal often helps a ton in my experience.

Also, her decision was not yours to make, so do not let guilt consume or poison you. You do what you can, and that is enough.

You might also find it a huge burden off your shoulder to just talk to someone about your feeling of betrayal and just let it out. Your father or boyfriend would be a good option, but first you might sternly tell them you just need their ears so you can let your emotions out, not their advices or solutions. Counselors or mental help professionals are also surprisingly decent methods to clear your system.

I wish you best.