I have an ex who did this EXACT thing. Hand always touching himself when we were alone. It didn’t start off that way but the longer we dated the more I noticed it was happening all the time. Just lightly fondling himself under his pants.

When I confronted him about how it made me uncomfortable (and like you, it made me very anxious and upset) he told me he would not stop. It was nonnegotiable. The frequency remained the same after our talk.

It’s not even like I have an issue with my partner masturbating. It was a boundaries issue and a blatant disregard for my feelings. I didn’t want to be constantly subjected to that and it was never consensual. There were other big issues and I broke it off with zero regrets. It took me a long time to get over the anxiety, too. Whenever I’d be with someone new I’d be afraid they would start touching themselves.

Later I’d find out he had issues with porn and chronic masturbation. I hope he was able to get help with them. For me, though, I know much more peace being with someone who does not do that.