This is actually a more fragmented shift than the question presupposes. There wasn't a point where you could say they all fit in either of these two categories according to most Americans. In other words, there's a slight problem with the premise.

In the late 1800s after the Opium Wars and the lead-up to the Chinese Exculsion Acts, the Yellow Peril was in full swing, exemplified in things like Sax Rohmer's writings and the sort of rhetoric that was developing against Chinese on the West Coast. But it did not generally apply to Japanese. At the same time that Chinese were being marginalised with attitudes toward them becoming negative, Japan was still fetishised in a popular light, and just after the turn of the century it wouldn't have been at all uncommon for the average white American of means to engage in things like sushi parties.

While the situation for Japanese wasn't the same everywhere, most notably in Hawai'i where there was a stronger resistance in the late 19th century, it wasn't really until well after the Chinese Exclusion Acts were at their height that the tables turned on Japanese immigrants, having been somewhere between tolerated and accepted up until then.

So that's one end, the Yellow Peril end, of the spectrum you've proposed.

For the other end we can look at the 1980s and 1990s where Chinese immigrants, now quite well established, were seen in terms of the "model minority", but again this did not apply to all East Asians. Koreans and Vietnamese still had significant difficulties, though where they fell on the scale fluctuated with time and place. During the LA riots, Koreans were very much not seen as the "model minority", and likewise Vietnamese immigrants/refugees in the 1980s and 90s were accused of all the same claims that they were unassimilable with America's culture that the Chinese had faced a century before.

The divisions between East Asian groups, as viewed by White America, were always more salient in the public eye than their shared Asian-ness. You can see this most strikingly comparing characters of Chinese (and Japanese) near the beginning of the 20th century with the sorts of WWII-era illustrations, for example "How to tell a Chinese from a Jap" where now the Chinese are described as more "normal" in comparison.

Attitudes toward different East Asian groups have been constantly shifting for as long as there have been Asians in North America, and chances are that will continue to happen.

It's late so I've not included a lot of specific details and references here but if you want to know more about any of the above points, please let me know and I'll be happy to fill in more detail as requested. It might take a couple days though because I'll be travelling internationally and I'm pretty sure my planes won't have wifi.