Why is the Legend of Zelda series so popular and why don't fans want Nintendo to adopt modern gaming ideas into the series?

You write this under the apparent assumption that modern game design ideas are universally "good". This is not necessarily the case.

Basically, all of the old Nintendo titles are gameplay first. Back in the day when you had 8-bit Mario jumping around the screen, it was basically impossible to sell a game on story or characters. Way back then, games were little more than core gameplay loops and highly stylized graphics (since it was the only way to even make things recognizable given the tech). For better or worse, Nintendo titles have kept to this idea throughout the years.

It's not as simple as "adding a story or character development" since these things can actually get in the way of the core gameplay. Take Metroid: Other M, for example. This game is widely considered to be a terrible Metroid game because it took "modern" cinematic storytelling techniques and character arcs and used them to railroad a game series about exploration and mechanical experimentation. To this day, Super Metroid is considered to be a hallmark of speedrunning and sequence breaking due to its freeform design and emphasis on nonlinear exploration, so the more modern approach of holding the player's hand through a cinematic experience was actually a detriment to the formula.

That's not to say that there's no place for modern storytelling concepts in these games. Despite being a little heavy handed at times, Skyward Sword is actually a pretty good example of this. Link has personal motivation for completing the quest for probably the first time in the series, and one of the side characters (Groose) actually turns out to be a real three dimensional character who undergoes meaningful character growth as the events of the game unfold despite initially seeming like a one-dimensional stereotype. He wants to be the hero for selfish reasons, but he has to come to terms with the fact that he isn't the hero and find his motivation to still step up and support Link against the impending threat. It's actually really well done. Stuff like voice acting honestly isn't really critical to the experience, as the character-unique garbled sounds serve the same purpose. Contrast that to another Nintendo franchise, Star Fox, which has incorporated voice acting since the N64 since it added to the experience by letting the player focus more heavily on the gameplay without having to stop and read dialogue in combat. In both cases, the justification is gameplay-first.

Beyond that, the Zelda games do change. There are basically 5 main 3D Zelda titles: Ocarina of Time, Majora's Mask, Wind Waker, Twilight Princess, and Skyward Sword. OoT, MM, and WW are incredibly different, and TP did receive heavy criticism for trying too hard to copy OoT. SS hybridized a lot of elements from previous games and mostly tried to innovate with the control scheme and combat (with arguable success). The next Zelda looks to be a proper open world game, which we haven't really seen yet from the series. It definitely changes far more than COD or AC from one iteration to the next, and it is not above criticism when it doesn't.

To be perfectly honest, if you're looking for story or narrative content, Nintendo really isn't the place to be. I noticed that you specifically listed games like Mass Effect, The Witcher, and The Last of Us, which are all very story-centric titles. Nintendo games have always been gameplay-first titles, so it sounds like you might be running into a personal preference... which is fine! You said that you're relatively new to gaming, so let me make something clear: no one game is for everyone. Find the games that you enjoy, figure out what it is about those games that you like, and use that information to find other games that you want to play. For example, I think Dark Souls is one of the best games ever made, but I wouldn't casually recommend it to anyone as a must-play due to the sheer difficulty. Deus Ex is another great classic that I love, but I'd be delusional to think that most modern gamers would be able to stomach that game's painfully visible aging.

Don't let other people tell you whether or not you should enjoy a game, and understand that different people will want different things from their gaming experience.

/r/Games Thread