The only things I avoid are unadulterated real-world references (e.g. spartan, Achilles' heel, lesbian, eau de Cologne, Wiener schnitzel etc) and English wordplay/puns. It is impossible to completely remove the covert real world references from the language (copper comes from the name of Cyprus, spruce comes from Prussia, to name two examples).

Plus, no language is totally pure: every language in every world would have imported words at some point or other during its history. I was surprised reading the other day about how many otherwise mundane words come from Portuguese, for instance. So I feel there's no point trying to get too far down into pure Anglo-Saxon -- certain Latinate and French words convey extra nuance (meeting vs rendezvous, deja-vu describes a particular feeling of familiarity and so on, caramel -- one of those Portuguese words -- is a particular sort of sweet; even champagne pinpoints a particular class of sparkling wine), and so on.

So...assume the characters are speaking in whatever language is common to them in the book, but don't forget, even Old English had drawn from different sources by the time the Angles, Saxons and Jutes settled down, and mediaeval England had sustained contacts with most of Europe plus the countries along Silk Road. Before that, the Celts and Romans had been here, and we've played host to a lot of different peoples over the centuries: it's thought that a third of the Scottish population has Flemish ancestry from a large refugee population during the various religious wars following the Reformation.

Other languages absorb their own surroundings: Russian borrows heavily from Turkish and the languages of Siberia. German developed slightly differently when that country split into two after the war, and Austria and Switzerland have distinct strands, just like Irish English has structures in it that are literal translations from Gaelic and Lallans- and Ulster-Scots have structures which resemble English but adapted to the cultures in which they were spoken. (I am a good mimic -- I stayed with my cousins in Northern Ireland for the summer holidays one year when I was a child and by the end was writing using dialectical words my cousins used.)

I really can't stress enough that you'll drive yourself potty trying to strip away all the non Anglo-Saxon words from English, and end up with something that you can't adequately use to convey a good story to your reader. I just assume when I write my story that if I stay away from modern slang/speech patterns and blatantly mismatched cultural references then I am free to use the words that my readers will understand in the way I intend them. (I can get a pass because I write steampunk fantasy, so have researched some Victorian speech patterns for my stories, but if you try to write in authentic mediaeval English you'll end up with something like Chaucer or Beowulf, and your casual readers will baulk at it, so it's always going to be a balance in that respect.)

Language is a fun thing to think about but you're writing to be read. There is no completely pure strain of English that would be something your readers could actually enjoy. Just find a strong register which conveys character and setting but keep on the right side of intelligibility.