or they're so familiar with the ins and outs that they're able to get away with using it in a way you never would.

This is usually the problem, but in Bethesda's case it wouldn't shock me if a LOT of complaints are levied by the testers and that they are IGNORED by management. I have not worked at Bethesda but I have worked in other large game companies and sometimes they have a LOT of trouble trusting their testers regarding game balance and flow.

At one place we'd basically mint a new focus group every month before release, just so they could tell management what we've been telling them for SEVERAL months. And as soon as those questionnaires came back, shit would finally start to get fixed.

Sounds pretty dumb and inefficient... but then I've also worked at a place that didn't do any focus group testing and just ignored everything said by everyone. (Including waving important bugs.) So still better than the alternative!

Bethesda just... feels like it has one of these problems from the outside to me. Again, I haven't worked there and this is just speculation based on comparison.

THAT SAID! Even if everything is going right at a company, it is VERY challenging to see the "big picture" when testing a game. Testers try very hard to detach themselves from what they know in the game but they are playing through it like once a week per person. They are gonna develop strategies and patterns that DON'T look like normal players.