It's not just about network speed. It's also about processing the data as it's downloading. My maximum download speed is 130Mb/s, which is reasonably fast but not extreme for where I live. A 65GB blockchain would take 1h34m11s. Try it, here:

http://www.download-time-calculator.co.uk/

The reason that it currently takes about 4 to 5 hours on a good PC is that there is computation involved. In 5 years time, let's say that I have a CPU, RAM and SSD that is a lot more powerful than the best available now, but for some weird reason (just like your reason that it's just about download speed) I still only have 130Mb/s. Then I could download and process the blockchain in, you guessed it, closer to 1h34m11s.