They refer to the mineral content of the water. A lot of the UK sits atop limestone or chalk (which is a kind of limestone), which is calcium carbonate. In these areas the drinking water has a lot of calcium in it, which tastes a bit ‘off’ (not neutral) if you aren’t used to it, and also gunks up your appliances (such as the kettle for instance but also washing machines, dishwashers, showers…) with calcium deposits called limescale. Using a britta filter makes a noticeable difference in these areas, which are said to have ‘hard’ water. Areas with lower mineral content in their tapwater therefore have ‘soft’ water.

Hard water can also refer to water with a high magnesium (salt) content, such as some places on the continent whose water is so salty as to be undrinkable, but in the UK it refers to calcium.