This is how I read the situation.

Comey's overarching goal was to maintain the integrity of the FBI in the public eye. Especially in the situation of the election, where people were increasingly angry at the establishment (Clinton being the surrogate of it) and there was an anti-establishment candidate which had alarming behaviour. One which was not above attacking the intelligence services themselves.

The fact that he could talk about the Hillary investigations means he did not regard it as an issue of national security. If he did he would have to give a GLOMAR response.

So he made it seem like the investigation was legit in the eyes of the Trump supporters to maintain the FBI's integrity whilst knowing that it will never amount to anything.

He then cleared Clinton but found out about the leak of the emails, and must have at least had some cause to suspect foul play about the leak. To maintain FBI integrity in the eyes of Trump supporters he then quickly did the whole letter fiasco before the emails dropped as the leak was a direct attack on the integrity of the previous investigation, which is a question of national security.

He then quickly cleared Clinton again but now had a new, but much more serious, investigation surrounding Trump.

During all of this he was denied warrants to look into Trump and Russia ties, he was only given them after the election.

So where we are now is he has a serious investigation which he cannot talk about.