It's very gradual.

Theon vanishes without a trace at the end of the third book, when the Boltons infiltrate Winterfell (after Theon had just done the same thing). There is absolutely zero mention of him in the fourth book. In the fifth book (which came out 13 years after the third one), we are suddenly given a brand new POV character called Reek. His very first scene is him eating a rat raw inside of a prison cell. In flashbacks, it's revealed that Ramsay Snow has been abusing this person (whoever he is) for a long time, including forbidding him from bathing, flaying, genital mutilation, and a false escape designed to crush his hopes. The first Reek chapter doesn't give us much besides this.

The second Reek chapter is when the hints start dropping, the most blatant ones are that he is called "turncloak" by some other characters (meaning "traitor") and is described as a prize that Ramsay has won at some point in the past. He also has vague memories of Winterfell from a time before, but he quickly subdues those memories as they belong to someone with a different name.

By the end of the third Reek chapter is when he is assigned to clean up to go to Moat Cailin and demand a surrender from the ironborn on behalf of the Boltons. It's in this chapter that Roose finally calls him Theon Greyjoy, but most readers realized his identity before that moment.

Imagine reading about this Reek character when it's been 13 years/1.5 books since you've heard a single thing about him. The slow realization of who this appallingly broken person used to be and just how far gone he is ends up being absolutely chilling. It's one of GRRM's most effective long-cons in the series.