Java is a programming language which talks to a virtual machine, this allows it to run on various hardware platforms. If your platform (i.e. Win/Mac/Linux/Android/Microwave Oven) has a Java Virtual Machine (JVM) written for it once, your platform can now run most if not all Java code with very few modifications.

So running Java code typically means each Java program talks to hardware like this:

Hardware <--> OS <--> JVM <--> Program

Ever wanted to know why Java is slow? It's because of this extra step to the JVM every time you interact with a program. This chain gets slow quickly once the number of instructions in a program gets bigger and bigger.

Under Dalvik, when apps were installed, they were downloaded as Java executables, and translated on the fly as instructions were passed. This is very slow, and very inefficient on mobile platforms where performance and battery life matter a lot more. They eventually got faster with JIT (just-in-time) compiling which improved this performance by intelligently translating most features of a program before it loaded. i.e. loading the next level of a game before reaching it, etc.

ART is completely different. It is an AOT (Ahead of Time) compiler. Rather than translating through the JVM, it fully converts apps to native code, so the new relationship upon installation:

installed app (Java program) --> ART compiler --> Native Android app

So now under ART, apps do this when executed:

Hardware <--> OS <--> Native app

This is significantly better, but it's not perfect. There are still a lot of things that need to be ironed out by both Google and developers to make everything work smoothly. Lots of apps are kind of "hacky", and use some Java techniques that don't quite translate well to ART. When they do, we will see much better performance:)

sorry for the wall of text.