Freedom of movement is for all the EU 28, the difference between states in Schengen and those outside being that you need to show an ID when you're travelling (the horror).

Not quite.. Freedom of movement isn't related to the uniform Schengen Visa. You can't enter the UK or Ireland on a Schengen visa (if you are from a country that requires a visa for the UK or Ireland, you'd need on in addition to the uniform visa..) nor do you have any other rights related to freedom of movement in the context of the single market.

There is a common Visa policy, it applies to Schengen not to the whole of the EU. Moreover, some countries have more, or less access depending on the agreements they have with EU states (visa issuing is a shared competency..). To give you an example, a Jamaican can enter Germany without needing a visa, however if they want to enter Bulgaria, Croatia, Estonia, Latvia, Lithuania, Romania or Slovenia (some of which, you will note, are Schengen area members, like Germany..), they would need a visa.