It gets a reputation because it does things that "for dumb people" entertainment doesn't do. Much of its humor comes from knowing the characters and how they'll react, some fans refer to this as running jokes, but really, much of the humor involves putting characters into a situation and letting the audience tell themselves the punchline.

Can you imagine big bang theory having well developed enough characters and trusting the audience to leave the punchline as an exercise for the viewer?

Arrested development also had some other things going for it that shifted the audience, like the show being a metaphor for how the bush administration was running things. Which famous figure was Charlie from 2 and a half men supposed to be? Charlie Sheen?

There are shows that lay everything out on a silver platter for the audience, and there are shows that trust the viewer can take the initiative to connect the dots and get a bigger payoff when they do so than if it was done on the screen for them.

Now "Smart people" and "Dumb people" can enjoy a show of either type, but shows that challenge the viewer to work a little for the payoff tend to be shows classified as "for smart people" and shows that run through the joke from starting pistol to finish line tend to be classified as "for dumb people".

Relax, it's just shorthand because big bang theory fans lose interest when you stop poking smart people with a metaphorical pointy stick.