I'm not a scientist, but I think it has something to do with dopamine and novel experiences. From what little understanding I have, our brains evolved to seek out new things; we would get a rush every time we came across a new type of fruit or saw an elk walking across the plains. Even information in a social sense had a purpose for our survival; who were the altruists in your group vs. the selfish people; who was cheating on whom.
Distraction also ensured our survival if there was a lion approaching the camp when we were busy eating our kill. If we were so caught up in eating to not see the lion coming, we'd get killed.
But, now days, there is too much information, too many things to do, too many people to follow.
Things are demanding our attention left and right. Our attention is also a precious resource. You have a day's worth of attention everyday. What do you spend it on? What thoughts do you think day to day, hour to hour, minute to minute? What you think about is what you act on.
Take porn and masturbation for example. I am regular reader of posts at /r/NoFap . There are guys (and sometimes girls) fighting over there with their own urges. You see posts from people that have abstained for years (from masturbation and porn, not always sex), to people that can't make it past day 2.
I am one of those people struggling to overcome that addiction. I have found out that if I can change the thoughts I think day to day, minute to minute, I can overcome this. It is only when I become complacent and let my mind wander am I overcome with the urge. Essentially, one thought (say the ass of that girl you saw on the bus this morning) leads to another (what would she look like naked), to another (lets see if I can find a porn star that looks like her), to another (ah shit, I just relapsed).
So, I think that the first point of defence has to one's thoughts. It is also the hardest to accomplish. It requires not giving your attention to any and everything that demands it. It starts with being mindful of everything you think, second by second. This is not easy.
Another thing that you can implement is the control of your environment. If your living, working, sleeping area is chaotic, it may be a reflection of your mind, or it could be that the clutter is the one making your mind chaotic. So, maybe drop by /r/declutter and /r/minimalism . These are tools you can use to become more effective. They're not ends in themselves.
Think about eliminating junk media as well: most of reddit, your television, sites with little content (Buzzfeed, Instagram, Facebook, Twitter). These are things that bring no value to your life. Most of the news to you see these days doesn't matter. If there is something really important, you will hear it through someone else. We're living in the age of narcissism. Everyone wants to tell the world all the good things about themselves, and they're quick to get upset if not a lot of people are not interested.
Sorry for ranting. I'm not just talking to you. I'm also trying to find an answer for myself as I'm typing this. This is a deep problem. There is nothing superficial about it. It affects my life on a daily basis. Every thought I spend thinking about the past, reddit, youtube, or anything else that doesn't get me closer to my goals in life is a waste. Really, it is a waste. I'm 28 now and think about to how much time I spent on meaningless jobs, video games, online chatrooms, useless articles, and people I thought were friends. All that energy that has been taken away from me that I will never get back. But, I think about the potential that energy and attention had. The things I could have done. I could have been in a better financial position in life, not living in a basement looking for jobs on Craigslist.
This point in my life is a wake up call for me. People always say that you will regret the things you didn't do when you're laying on your deathbed. I took the most lowest meaning of that. I applied that logic to the most useless of things. At this point in time, I regret the waste I have made of the past 28 years of my life. That dopamine reward craving, whether that comes from reading that funny joke on reddit, jacking of the the newest video on xtube, or doing things to social acceptance makes a slave out of us.
Ideas:
Establish goals in life and give them time frames. For example, buy a notebook and have a page dedicated to things you want to accomplish in one month, in six months, in a year, in three years, in five years, and so on. If you don't know where you're going, others will decide for you.
Be passionate about those goals. Maybe, once you have made the lists of the goals, read them and contemplate on them after waking up and before going to bed. Get your mind thinking about how you're going to get where you want to get. Get emotional about them. Imagine the sense of accomplishment you'd feel. If would be sweeter than any cat gif and you'd have more to show for it.
Break down these goals into smaller tasks, and then even smaller ones. Write these individual tasks down and hold yourself accountable for accomplishing them.
Be careful about who your surround yourself with. Not every one is worth spending your time with. Are really are the product of the people you associate with. The average reddit user and the average engineer/astronaut/intellectual do not talk about the same things. They do not spend time doing the same things. Think about the kind of things the people you're with do. Are they in line with where you see yourself going in the long term? If not, maybe don't get so caught up in pleasing them. If they're doing things that you do not see yourself doing in the next 3 years, and they show no signs of ever wanting to grow up, maybe try to distance yourself. You don't need to be Machevellian about it, but really, take a long hard look about who they are at their core and where their lives are going.
Go into depression about your wastefulness. I recently did this with all the money I'd spent on things in the last 8 years. I opened up a spreadsheet on Google docs and tried to recount every item that I spend money on just because I could. From beer and pot to backpacks and sleeping bag for a round-the-world trip that never happened. Around $30,000 I had just thrown away on essentially useless things. It was a real eye opener. And I'm glad for the experience. It has made me much more conscientious of my daily spending. I'm much more frugal and now try to get the most use out of everything. You can do similar things for your friends, jobs you've held, courses you've taken, etc. For example, you can create a list of all the people you've know that you didn't really like and next to it put a rough estimate of the amount of hours you've spent with that person. Add it all up and look at the waste. Now take this regret and turn it into something useful. Be careful about who you spend your time with.
Work on your surroundings. Eliminate unnecessary things. I'm not going to rant about this one. There lots of people doing that at /r/declutter and /r/minimalism
I'll end this by remaking the point that I feel is the most important one: it really comes down to your thoughts. What do you spend time thinking about? Is that thought contributing to a future that envision for yourself with you living at your highest potential.