The difference is that vinyl is an analog recording or sound, which is the audio in its full, original state, and CD's/digital is, well...digital, which is not audio in it's original state.

Digital is basically snapshots of that same audio glued together to recreate it in stunning accuracy. However, it is not as lossless as vinyl which is about as close to a full capture as there is. So, vinyl has fuller sound while digital will always lose some of the finer details that vinyl (analog) is able to hold onto. Honestly, the average listener probably can't even tell the difference a lot of times, but if you are an audiophile to any degree there is a stark contrast.

The catch is that in the original era of vinyl, audio technology wasn't nearly at the level it is now, so there is a limit to how well any recording from that time can sound. Technology has advanced monumentally since then, and recordings of new music on vinyl will reflect that. That being said, vinyl can break down after a lot of use, so it's not the most durable material for longer periods of time in which you listen many, many times over a few decades.