I used to feel the same way, but you don't realize something about it. A lot of guys you see with hair have receding hairlines, because it starts in a very subtle way. When you think "going bald" you generally don't notice the guy who has it long on top but his hair has receded a half an inch or an inch, but he leaves it long on top to fool you.

He notices it, but you don't. At least part of the reason why the long on top style persists is because it makes a slightly receding hairline look perfectly fine.

When you can see the top of your scalp though, yeah you'll shave it, but there's a lot of room between there and the hair dissapearing from your temples.