The time between Oblivion and Fallout 3 was more like 2.5 years, as the games were released in March 2006 and October 2008, respectively. Actually, Oblivion was planned for late 2005, but had to be delayed by a few months. Without that, it would have been fairly consistent 3 year gaps until Skyrim.

On the other hand, the development of newer games is increasingly expensive in terms of resources and amount of work required. In the PS3/Xbox 360 era, 3 years of development by 100 people was enough to make a competent AAA game, but nowadays there are large open world titles created by studios equal to or larger than BGS even after the recent expansions, and they take 5 years. I think it is no accident that Fallout 4 already needed 4 years after Skyrim, and even then it feels somewhat under-developed. That is why it seems doubtful if TES VI could be finished in only 3 years after Starfield, considering how high the expectations are after such long wait.

Three years between Fallout 76 and Starfield is more plausible, since there was at least some level of parallel development between those titles, and the latter could be fully focused on by the main studio sooner than usual, because post-launch support of FO76 is handled mostly by BGS Austin.