I've always liked Adam Koralik's (YouTuber) take on the 4th gen. Sega and Nintendo I'm that particular generation battled it out, and the only way to win was to just make the better freaking game. Even titles like Beavis and Butthead, for example, were drastically different on the two platforms, trying to develop to what people would want to play. They were dedicated gaming companies that needed quality to win.

Now, Sega is gone and Nintendo is around as the #3, but still releases underappreciated quality. Sony and Microsoft sell features and exclusives, and while there are tremendous achievements made on both consoles (The Last of Us is a masterpiece that only got better with the PS4 remaster).

Anyways I think it was around the 6th gen when Sony used the PS2 as a DVD player to sell units that I guess the passion fell away from major developers. Companies can just patch a game whenever which is great, but to release something like Assassin's Creed Unity, how broken it was, wouldn't have happened from a major publisher I don't think back in the 4th generation, for example (or maybe it did and I have rose coloured lenses). So I think as OP mentioned, hardware made it so games had to be the best they could be on day 1, and there was a desperation and drive in most products of the olden days. There's also the life vs. Art debate, that a lot of games, while still fantastical, are basically ultra realism compared to classic games that didn't have the confines of hyper realistic graphics to operate in. I think you can pop in an old game and just have a release when you get sucked into it. Most modern games are life simulators with a fantasy element at play (whether you're a witcher or an LA criminal), and old games felt more like games? If that makes sense?

Anyways sorry to ramble.

TL;DR - Older gens had to make the best game they could with the tech and the competition, whereas now there's so much extra it doesn't always feel like a game anymore.