Its really not surprising. We've had some big games release since then that are much easier to get into with a group of friends (overwatch and pubg to name two popular ones).
Dota requires quite a commitment if someone wants to play at a passable level, and is just a massive information dump as you get into the game even with a limited heroes pool. It lacks a proper introduction to the game and relies on the player either looking up how to play from external sources or being informed on how to play with a friend.
I think Dota is falling into the same bubble that Sc2 fell into (not in terms of playerbase, but rather who the game is aimed at). People that play Dota more often than not want a competitive experience. Dota really isn't the game that you mess around in with a bunch of friends on a Sunday afternoon. You have other games for that now, which are simply easier to get into and don't have such a steep learning curve.
Now, all these points that I mentioned don't have to be negatives. Dota is great because it caters to the right market, but that will lead to other games potentially over taking it in numbers. It isn't the end of the world, and eventually the numbers will plateau. Aslong as that number is big enough to sustain the game, we really don't have anything to worry about. I personally don't need to see Dota as the most played game on steam. Just because its losing out players, doesn't mean its losing out on its core playerbase. Every game loses players over time, as new games come along. That's just the nature of the beast.