Your world is about to get bigger... a lot bigger. In CS, the military are the most important factions. And by missing 5 games, you don't develop a strong connection with the actual important factions: Bracers, Ouroboros, and the Church. You missed 5 games and things like the Sept-Terrions weren't even mentioned in CS1/CS2.

Besides Olivert/Olivier, another connection you will miss, is how they hype up the "Eight Leaves, One Blade" school in all games, because the stronger characters in the games, the Divine Blade and Master Ka-fai, are adepts of this school; but you already know what it looks like, because of Rean.

It's still worth playing though. You won't regret it... there's a lot more to the world than Cold Steel implies.