There are a lot of variables to consider.
Due to a weak developmental system, American players are often late bloomers. Guys like Geoff Cameron and Clint Dempsey were nowhere near Prem ready when they were 20-21. When you look at an American player, at least from the past generation, you could usually safely assume that he was a couple years behind his age level European/South American peers in development.
With that in mind, remember that Altidore was a kid at Villarreal and Hull (and also remember that Hull were shit regardless of him). He was younger than Oxlade-Chamberlain is now and green as grass. Honestly, he was not very good around the time of the 2010 World Cup. It wasn't until he got to AZ that he really started to shine for club AND country. What a lot of Europeans probably don't realize is that there were (and still kinda are) A LOT of Jozy haters in the USA around 2010-2011. People who were itching to see him out of the lineup. He had a tendency to disappear from games (sound familiar?). But he stepped up his game and it became clear that he could thrive in a system that utilizes his strengths (he is good at combo play and facing the goal -- pretty shit as a ball-winning target man).
Most of his European clubs have been bad. Sunderland and Hull were among the worst in England during his time there. You can't lay that all on him. Very weak supporting casts. He is not the type who can score on his own. He's more of a role player than a one-man army. When you get guys like Donovan, Dempsey, and Bradley or Bereens, Elm, and Maher working with him, you see him become very dangerous. When you stick him on a squad full of selfish and/or bad players, he reliably disappears. I don't think Sunderland/Hull were all to blame for his struggles there, but you can bet that he would have benefited by having better facilitators and scorers around him.
He consistently plays well with the national team. This is probably the biggest thing above all else. There are some players, like Gio Dos Santos for Mexico, who play well in the national shirt even when their club career is looking terrible. I wouldn't expect a Mexico fan to think Gio is shit just because it didn't work out at Spurs. Likewise, you can say that Altidore sucks, but if I see him consistently playing well against reasonably good competition for the USA then I don't need to agree with you. People will say it's "just friendlies" or that "CONCACAF sucks" but it's simply not that easy to succeed at the international level. He has performed well against numerous teams full of players from top 4-5 European leagues.
Big-ish clubs still rate him. Teams like Fiorentina and Lazio were sniffing around when he signed with Sunderland. He could've gone to Germany, France, or Italy when he left Sunderland. It's not a case where Toronto was his only option.
I understand why foreigners would get sick of hearing about the USA on r/soccer, but at the same time from the American perspective we don't really get the respect that we deserve at times. Foreigners are really reluctant to recognize any of our players. Donovan goes over to England twice and wins Player of the Month both times for Everton and yet people still can't accept that he could've been good enough to get caps for England. They just love to hate on our team and our fans and pretend we're all clueless. Reality is that many hardcore fans in the USA have a very accurate understanding of our players. We know that Dempsey and Bradley are not Bayern material. At the same time, it works both ways. We know that other players like Altidore and Fabian Johnson are actually quite good regardless of the fact that fans in England or Spain might not think anything of them.
From my perspective, Altidore's level might be similar to someone like Bendtner. Not a top player. However, not the useless pile of crap that people make him out to be. Both guys have washed out of some club situations, but have also done fairly well for decent national teams.