If self-driving taxis were competitively priced and always available I'm sure many people would opt for those over owning a car.

About the driving is fun bit, I was actually confused when I read that as my wife and I both hate driving. The romantic idea of driving to ease your mind or gain a sense of freedom is nothing more than a delusional fantasy for me. I don't doubt it's a different experience for you, I just personally have a hard time relaxing while doing something so demanding. I spend most of my time driving around Seattle which could explain our differing opinions, do you live in a rural area?

For me, driving is very frustrating, boring, and the reason I've always wanted a self-driving car - dangerous. My life is entirely in the hands of other drivers for several hours each week. I consider myself an exceptionally cautious driver, yet I still have multiple close-calls every month due to other drivers errors. Every time I get behind the wheel I know there's a big risk of getting in a wreck that injures my family or myself, being thrown in jail or forced to do community service(happened to my wife even after witnesses testified it wasn't her fault), or owing somebody a lot of money in damages. There's of course other annoyances like being late after a random checkpoint or police encounter, missing an exit because some asshole was flanking me, other drivers going 20 under the speed limit, couldn't find a parking spot, and wrecks which would all be very uncommon if everyone used self-driving cars.

Now the facts. Several studies reported between 90 and 99% of wrecks can be attributed to human error. Automobile accidents in the US cost around 871 billion per year, which is more than many estimates of the cost of universal healthcare for every single American over an entire decade. Based on 2010 crash statistics, nearly 100 Americans lose their lives each day, while more than 6,000 are injured. Self-driving cars are much less likely to make the errors that contributed to these statistics, and obviously don't come with the risk of being impaired from drugs and exhaustion. I find it disturbing we haven't publicly funded or implemented this technology by now. Even something as simple as a $20 device that beeps or lights up when it detects upcoming active stoplights could prevent thousands of deaths, yet we're more worried about regulating how much ppm carbon a Ford Focus puts out.

A few sources:

http://cyberlaw.stanford.edu/blog/2013/12/human-error-cause-vehicle-crashes

http://en.m.wikipedia.org/wiki/List_of_motor_vehicle_deaths_in_U.S._by_year