I just finished reading Guns, Germs, and Steel and this is one of the key questions Jared Diamond tries to answer at the end. His take is that it all comes down to geography:

"To understand China’s loss of political and technological pre-eminence to Europe we must understand why China has remained united, and Europe fragmented. Maps provide the clue. Europe has a highly indented coastline, with five large peninsulas that are island-like in their isolation, and all of which evolved independent languages, ethnic groups and governments: Greece, Italy, Iberia, Denmark and Norway/Sweden. China’s coastline is much smoother, and only the nearby Korean peninsula attained separate importance. Europe has two islands (Britain and Ireland) large enough to assert their political independence and maintain their own languages and ethnicities. Indeed one is big and close enough to become a major independent European power. China’s two largest islands, Taiwan and Hainan, are each less than half the area of Ireland; neither was an independent power until Taiwan’s emergence in recent decades. Japan’s geographic distance from the Asian mainland kept it much more isolated politically than Britain has been from main-land Europe. Europe is carved up into independent linguistic, ethnic and political units by high mountains (the Alps, Pyrenees, Carpathians and Norwegian border mountains), while China’s mountains east of the Tibetan plateau are less formidable barriers. China’s heartland is bound together by two long navigable river systems in rich alluvial valleys (the Yangtze and Yellow rivers), and it is joined from north to south by relatively easy con-nections between these two river systems. As a result, China very early became dominated by two huge geographic core areas of high productivity, which gradually fused into one. Europe’s two biggest rivers, the Rhine and Danube, are shorter and connect less of Europe. Unlike China, Europe has many scattered “core” areas, none big enough to dominate the others for long, and each the centre of an independent state."