Not to be rude, but if "Your haircut frames your face really well" is the most inappropriate thing you can think of that he said to you, I've really got to question how you're looking at the world, because that is about as innocuous a compliment as I can think of. If there's more to this that you didn't share and he was making inappropriate sexual advances/gestures/contact, then that's something else entirely. You said he was trying to read the text on your shirt--were you getting an overtly gross/sexual vibe, or did the shirt have a lot of text?

Also, regardless of his sexuality, and regardless of whether you reserve its use for extra-office use, I have no idea why you'd think 'gaylord' is a remotely acceptable thing to call anyone. I'm gay, and if I heard someone call someone else that, I would have absolutely no problem setting them straight--if you're using a word that describes a group of people as a way of putting down a different group of people, you seriously need to make some changes to your vocabulary, that is incredibly offensive.