The sound engineers (sometimes one person, sometimes a team depending on scale of the production), is actually running multiple mixes of the band at once.

The Front Of House mix (FOH) is what the audience hears, blasted out the front of the main PA.

The band, however sometimes like to hear a different mix to the audience in the interest of nailing their performance, and this usually delivered through a 'fold back' system - a series of speakers pointed back on stage towards the band.

The In Ear Monitors (the earpieces you mentioned) is a further development from the Fold Back system, where the musicians can get their desired mix delivered straight to their ears (which means you don't experience the sound change as you move around the stage / aren't adding extra noise into the room through a second set of speakers, and you can get the sound REALLY in there...).

The coolest part - is that you can then separate the Fold Back mix further, to an individual level - so the drummer can hear what he wants to nail his performance (ie have a metronome running), without hearing unnecessary things, and the singer can hear more of his voice than the others maybe want without stepping on anyones toes, the bass player can have the kick drum turned up really loud so they can keep in time easier etc.

A further development on that again, is that you can get mini-mixers that sit on stage, so each musician can dial in the mix they want themselves, without having to ask for changes.

As to why the play with them, it can feel much more natural having one ear in / out as you get the ambient sound of the room / people want to hear different things for different songs, and maybe the sound engineer forgot to turn up / down a particular instrument. Sometimes gear will malfunction (loose connections can make funny/loud noises in your ears).