See you would be right, except you're taking the larger stance of PC Gamers as people that use Steam essentially. That's not really the case.
The truth is that while we can see how many Bought a PS3 or an Xbox 360 we can't really determine how many bought a PC for gaming. Excluding the offline players, looking at Steam profiles doesn't really show how many play each game type so much as concurrent players.
The reason a few million is way too large a number is because that's beyond the niche approach. A few million might be the total number that actively play games modded and that's a very large niche, but in terms of actual PC gamers and then comparing that to the rate of adoption for new monitor and graphics card technologies, I think you'll find that the numbers don't add up.
Let's look into it:
Right now the concurrent steam users is 9,022,595 playing who knows what or just being online (because Steam starts in the background unless you turn it off and most likely don't turn it off).
Further, 27 million play at least one game of League of Legends every day. (A PC Game) 7.5 million players play during daily peak hours.
http://store.steampowered.com/stats/
The vast majority of these games being the 99 most played games on Steam concurrently have little graphical application or requirement.
http://store.steampowered.com/hwsurvey
This is an indication of the hardware trends. The majority, being 34.72 percent use 1080p monitors at the standard gaming resolution set. The percentages otherwise are low.
If we translate this statistically that means of the 9 million active players that only 3.12 million are using that monitor set.
That also means that if even 1% were using 4K it would amount to 90,000 people roughly concurrently. But we know that not even 1% are cumulatively using 4K. In comparison, all the 1440p combined equals: 5.24% roughly and this primarily centers on laptops. That means that 471,000 roughly are using 1440P displays in some capacity in a concurrent form. The most popular being 1440 by 900 which is a common laptop resolution. The knowledge of whether these are the same people using the 1980 but with more computers or just laptop gamers is unknown.
The multi monitor statistics are confusing we will skirt by them because that's a niche that's growing as well and it is irrelevant to this discussion.
Now, .07% are using Ultra Wide HD, among a few other strange resolutions. This is the highest indication of any significant resolutions. These numbers concurrently come to: 6,300 users concurrently rated as having those types of resolutions for gamers that play actively on Steam.
And that's for Ultra Wide HD, not 4K itself which is more expensive, requires more things, and etc.
As such, if this is the rate of transition after 1440P and Ultra Wide has been out for some time and after graphics card have been getting more and more powerful and games more and more graphically enhanced, then it gives a very poor rate of progression expectation for PC Gamers.
This is a niche, and 4K is a niche of a niche.