I used to be scared. I had ugly duckling syndrome so the moment I hit puberty I was completely unprepared for attention. I've had some bad ones, from being punched into walls to being followed home to having them destroy my career by poisoning my reputation because they couldn't handle rejection. This probably doesn't put you at ease, my bad.
Anyway. I've found that the best method is to gross them out or scare them. I don't give two fucks if they go round gossiping about what I did because people who know me know exactly why I did it to them lol. Also this shit totally cheers me up over bad situations like yours. You can do all of this after the encounter, so that they don't want to do anything to you as revenge. Some ideas:
learn how to burp real good. I'm a motherfucking pro at this if I say so myself. This is great for if a guy hasn't technically asked you out, but is asking questions that lead to it and you know he'll call you a snob if you try shut him down. If he calls you pretty or whatever, burp loudly, maybe loudly wipe your nose with your hand in an upwards swipe and be like yo.
getting hit on when wearing sunglasses. Go cross eyed and then take them off. Maintain cross eyes and then repeat everything back really excitedly. OMG YOU LIKE ME? YOU WANNA DATE ME? OH MAN MY DAD WAS ALWAYS SAYING NO ONE WOULD LOVE ME, I CAN'T WAIT TO INTRODUCE YOU GUYS. I used to be able to just do one eye.
Get a cubic zirconia ring. My boyfriend bought me one to wear on my ring finger when I'm around the above mentioned guy who destroyed my career (still have to see him regularly ugh). Boyfriend doesn't even stand up for me if I get hit on, he just thinks it's funnier to sit back and watch lol.
my biggest peeve is being asked for my number/availability before someone introduces themselves. I tend to ask them why the fuck I'd give my personal information to someone who only cares about what I look like because they didn't even ask me how my day was or what my name is. Thanks for letting me know that my personality and intelligent isn't worth shit to you buddy. You'd be surprised how often I get an apology for that.
I think you already did really well by telling that guy it was none of his business. Such a great answer, and don't let that put you off standing up for yourself more because he would have done so much worse if you had given a non committal answer that he took for affirmation. If you're worried that he will make your life trouble, act as unattractive as possible so he forgets you.