I feel it can be helpful to mindfully consider these circumstances through the lense of sincerely asking "what is Good to do in this situation?"

Due to the unique nature of every interaction, in my view it is wise to consider the potential consequences of how you interact and orient one's intentions from there.

Are you enabling and thus contributing to a person's negative habits being cemented further due to your aversion to the potential discomfort of confrontation? Is that your intention?

Or are you being overly selfish due to your impulse to use confrontation as a tool to preserve your sense of self? Is that your intention?

Not saying either of these is the case, but these are examples of how you could question these circumstances in earnest.