Hey, fellow designer here. I'd like to give my input on your post. Naturally, all my comments about Nintendo's intentions are speculation - it's hard to know the true intent behind the design without asking the developers specifically. Heyo, fellow designer here. I enjoy pondering why Nintendo operates the way they do so I'd like to comment on your post. Note that this is all speculation as it's difficult to know the exact intentions of the devs unless they tell you.
Why are there no default formats that help guide people towards ideal rulesets?
At its core, Smash Bros is a sandbox platform fighter. Since its inception it has very purposely avoided declaring specific ways of playing the game, instead providing many options for allowing players to create their ideal experience. Ultimate is a good example as an enormous variety of gameplay options are presented. In one match you might have a 1v1 with no items on final destination and in another you might be presented with bomb-ombs only on triplats where the floor is lava. These are all presented as equally valid ways of enjoying the game, and that is completely intentional. I hope you understand that you are revealing your own biases when you claim that there are ways of playing the game that are more "ideal" than others. I've been playing platform fighters for twenty years and I've met people with wildly different expectations of how the game should be played. Pokeballs only. Final destination only. No "cheap" characters. The list goes on ad infinitum.
You can draw a comparison to other sandbox games like Minecraft or Mario Maker. There is a small, dedicated population of players that build literal computers in Minecraft. There is a small, dedicated population of Mario Maker that build levels requiring absurdly precise inputs and un-intuitive manipulations of mario physics. It would be ludicrous to suggest that these are the "ideal" ways to play these games as both games offer an enormous range of play experiences. But this is the mindset of the competitive smash community. None of these modes of play are bad; in fact they are truly wonderful. But it is important to keep in mind that these focused communities are part of a greater whole.
So you can see why For Glory/Fun was removed in favor of the Preferred Rules matchmaking as it allows people to play the way they want with people who also want to play that way. Personal anecdote: I hated For Glory. I want platforms in my platform fighter, I want at least 3 stocks, and I want to have enough time to murder my opponent without getting sucked into a sudden death match right before I'm about to close it out. And sometimes... sometimes I want to do this. You just can't get this with a system that tries to create distinct categories in a game where there are fundamentally none. If the core design is about letting people tailor their experience then you must create a matchmaking system that also accommodates that, or there is a jarring miscommunication to the player.
Why is the competitive scene so under-appreciated and under-serviced by Nintendo?
Why does it need to be? There are plenty of examples recently of developers that try to force their games into the spotlight by hoping that throwing enough money will create the next runaway esport. It doesn't seem to work out that well, and I find that it often feels very artificial and "hollow." Then when interest in the game wanes or it isn't as profitable, devs pull all the money out and players feel blindsided and frustrated. Heroes of the Storm and Artifact are good examples of this.
Why not let the game just stand on its own merit? If it's a great game people will want to play it, compete in it, and watch it. Melee is one of the most successful esports of all time and that's because the people keep it alive. Grassroots, baby. It's our love for the game; it's our scene. No one can take that from us, and it will only die when no one wants to play. Not when some corporate overlord pulls the funding.
And if we're being honest, it's not as if Nintendo hates us. When they hold events they bring in the players people want to see and give our favorite commentators a voice. Yeah, the tournaments are wacky - but you gotta remember they can't promote smash as having one "ideal" or "valid" game type. I think it's pretty cool of them to acknowledge the competitive style for finals like they did at e3.
Why aren’t there more setups in online Battle Arenas, so as to mitigate unnecessarily lengthy wait times for players?
Battle arenas are an emulation of local play. You sit around and take turns. You decompress during breaks. You observe the patterns of other players. You plot the demise of that guy who is ruling the room. You try not to lose because there is something on the line. These are all very intentional decisions that help foster a social experience. If you don't want to wait it's pretty simple to just make or join a small room.
Why is there no ranking ladder for GSP, or a means to signify its value and/or significance?
Because GSP has no significance. Since quickplay is not a pristine competitive experience, creating a robust ranking system that can somehow accommodate for all possible game mode variations is impossible. I believe GSP exists mostly for two reasons:
In tabletop design we often think about designing around the "chaos player." This is someone who isn't trying to win the game. It sounds kind of innocuous, but it can cause a lot of problems like stalling, sacrificing oneself to harass one specific player, or sowing other kinds of fun-dampening discord. GSP is a psychological trick to get most people to care about the game, even if it's just a little bit. It provides a wisp of an incentive to not goof around and create a frustrating experience for the person you're playing with.
It's something for players who are motivated by extrinsic rewards. Many people will play the game for intrinsic rewards like feeling accomplished at improving one's skill or just enjoying the game. But plenty of players exist who simply see no value in playing the game unless they get something out of it. You could use matchmaking rating for this, but most matchmaking systems are not designed to increase perpetually. If you are winning 50% of your games and losing 50% of your games, you are right where you're suppose to be and your MMR will remain relatively stagnant. But people who are extrinsically motivated don't like this; they want to see numbers go up. This is why there are so many progression systems in modern games that reward you just for playing. You may win a tough match in Quickplay and it's likely that your internal MMR didn't change much, but you got a bunch of GSP points and that feels good. It feels like a reward.
Whilst certainly not limited to Smash Bros Ultimate, but - why are there no chat or communication methods that allow gamers to connect with another and actually.. communicate?
I believe there are two Nintendo tenets at work here: don't alienate anyone and don't waste resources. Allow me to elaborate.
Look, the simple truth is that people get harassed for using voice coms. You can say "just block", "don't be a snowflake", "doesn't justify the rest of us suffering" all you want, it doesn't make it acceptable to expose people to the level of vitriol. If you're a hetero-normative white cis man you probably just don't get it. It's a pain, sure. But I want to play video games to enjoy myself, not roll the dice every time I log into coms to see if someone is going to tell me to go back to the kitchen. The abuse adds up over time, and I just don't want to deal with it. Splatoon is a great example here. Sure, it's frustrating when your team is fucking about in ranked, but no one has an advantage over anyone else because of the circumstances of their birth. And if you want to play a more coordinated game, then you need to engage with the community, find a partner whose company you enjoy, and get on the Nintendo app or discord or whatever. That system, quite simply, does not alienate anyone. It is a safe space. There are people for whom this is extremely rare and important, and I ask that you try to empathize.
The other thing is why go all out creating tools that others have done better? Blizzard dedicated resources toward putting voice chat in WoW and HotS and barely anyone uses them. Most people are just going to use their preferred solutions, like Discord or Ventrilo. This is why the Nintendo app is so bare-bones: it's meant for people who have no idea how to use coms. The rest of us will use whatever program we always use.
In general I think Nintendo hasn't done enough here, however. There should, at the very least, be preset messages that help you invite people to the games you want to play. It is disappointing to stick with someone in quickplay for a while, have some laughs, and then have no way to communicate with them if they friend you.