It's definitely possible. The only time I was ever arrested this happened to me.

I had cops execute a search warrant on my house, as well as friends house. There was nothing found at my house so they uncuffed me and let me ago. Two weeks later they showed up at my house with an arrest warrant saying I was being arrested for conspiracy. They hadn't gathered any sort of "new evidence" during this time, rather, they were just putting the justification together to claim that I knew about the criminal activities of my friend, even though there was no direct evidence tying me to it.

Basically yes they can arrest you later on if they can go to a judge or grand jury and argue that you were guilty of some crime based on the evidence they present.

If they can't find you, then there will just be a warrant out for your arrest until they do.