In theory, we wouldn't know it. It would create a paradox.

We assume that the cause of something always precedes the effect of it. We experience time to be forward-moving though, which is why the idea of an effect coming "before" the cause of it seems absurd.

If time is a static dimension in the same way that height, length and depth are, someone with access to a higher dimension than we exist in could "travel" across time but wouldn't be able to change anything. Perhaps a better way to phrase it would be that time in this context is like a photograph, where you can look at all the details within it but you can't alter it.

Let's say you go back in time to kill your grandfather. If he doesn't meet your grandmother, one of your parents can't be born and thus, you can't be born, which would obviously be required in order for you to have traveled back in the first place. So it might play out with you pushing him off a cliff, and he ends up only breaking his leg. In the hospital, he falls in love with a nurse, who happens to be your grandmother, and the events that led to your existence continue to be fulfilled.

tl;dr If someone traveled back in time, we wouldn't be able know they were a time traveler because that would fuck things up too much and certain laws of nature prevent things from getting too fucked up.