I virtually only read history books and you really shouldn't hope to remember everything. What makes things stick is repetition and you don't want to read the same book thrice.
What you should do is read more about semi-related topics and see the parallels/causes/effects, let history become a narrative instead of a list of names and places. (Also get your geography on point. Knowing where things are happening makes it more alive. EU4 and internet map quizzes helped me greatly with this.)
You'd be suprised how easy it is to retain facts when you can place them in a web of previously aqcuired knowledge.
I usually read a bunch of semi-related books after each other which really helps with this. It also happens to create a timeline in my case, which makes it more of a narrative.
My current spree started when I solotravelled to Florence and Venice: The Medici by P. Strathern, Il Principe by Machiavelli, Brunelleschi's Dome by R. King, Leonardo da Vinci by C. Nicholl, A History of Venice by J.J. Norwhich.
Napoleon: A Life by A. Roberts (He was the main guy in the closing chapters of those Italian republics, wanted to know more. Favourite book so far by the way!)
The Force of Destiny by C. Duggan (From Renaissance Italy and Napoleon to unification)
Iron Kingdom by C. Clark (Interested how German unification compared with Italian)
The Scramble for Africa by T. Pakenham (Another favourite. Colonialism was featured in the previous two and I got curious)
Dreadnought by R.K. Massie (Naval rivalry between Britain and Germany was alluded to and I wanted to know more.)
Finally: The Guns of August by B. Tuchman. This is where I'm currently at. Really liking it so far.
But linking all those stories together by reading them in succession and supplementing them with podcasts (Mike Duncan's French Revolution!) and games (Europa Universalis 4, Victoria 2) really made me appreciate and retain them more.
In summary: I treat history like fiction. Nobody enjoys just reading the Silmarillion, but if inserted into LOTR's narrative it can be quite neat.