The common denominator in all of this dysfunction is your boyfriend. He makes questionable ethical decisions in life, love and relationships. He is 53 years old so this is a life long pattern of his and he will continue to make bad judgment calls and bring unstable people into your life. I am thinking about the foursome here. It is just an example. If he had been a normal trustworthy guy with good judgment who cares about his girlfriend he would have vetted the people he brings into your life in one or another capacity. But he doesn't. I don't want to sound alarmist here but he is unsafe for you. I think this is why the other commenters have told you to run.

You said it yourself: "I find myself to feel cold for him recently for the choices of women he made and I’d like to keep myself away from unethical practice". Listen to your gut feeling. He doesn't vet people for your safety, so he is unsafe.