So while I obviously agree that you should tell him that he's making you uncomfortable, some of the responses to this post are downright fucking disturbing.
It's not true that saying "I don't want to date you, stop touching me" is the only "mature, straightforward communication" possible here.
What these people are suggesting is that women need to talk to men on a man's level - Confident, assertive, possibly even aggressive, because that's the only way you can expect him to understand. MANY times I have heard men suggest that women should be physically aggressive to someone who is assaulting them (OH BOY I can't wait to tell you how that usually turns out)
What others are saying is that men need to learn to pick up on strong clues which women leave to show their disinterest. To women, these signs are extremely obvious and we are very adept at recognising when another woman is letting someone down (I'm not saying mixed signals don't exist, but it's beside the point). Also, these are things which are conditioned into us in order to mitigate a potentially dangerous or emotionally charged situation. Until men are routinely taught to respect a woman's personal space and treat her like a person, women are going to continue to use signals rather than direct communication.
Whilst it's very possible that the OP doesn't actually feel physically threatened by a strong rejection of this guy romantically, her social structure is - Would any of you really trust that someone who shows affection by pulling a woman around, touching her, kissing her, is going to take a rejection maturely and politely?
As a hyoo-mahn fee-male who happens to wear clothes and earn profit, I'm tired of being told to speak and act like a man in order to be successful. Maybe it is time for men to change and learn the Language of Disinterest. It mostly involves a degree of self-awareness.