Potentially, yes, they can be harmful.

Inhaling anything that is not normal clean air is in general not good for your lungs. It doesn't matter if it's cigarette smoke, campfire smoke, coal dust, sawdust, finely ground corn starch, or paint fumes. It's stuff that's not designed to be in your lungs.

E-cigarette liquid (and hence vapor) largely contains either propylene glycol or vegetable glycol, both of which are sugar alcohols which can irritate the internal structures in your lungs and with long term heavy exposure can irritate and do damage to them. The heated coil can also break these compounds down into small amounts of other things -- burnt carbonized byproducts, etc., which have the potential to be harmful. The jury is still out on any secondary effects caused by exposure such as increased cancer risk, etc. only because the research has not been done. There is little to no casual evidence of people dropping stone dead from e-cigarette use, so it is likely the the damage and risk is therefore very, very small. There are much worse things you can do yourself than smoking a zero nicotine e-cigarette.

Heavy e-cigarette use, especially if you fully inhale, can lead to buildup of glycol and other byproducts on the insides of your lungs. In an ideal world your body can break down and get rid (most of) of these contaminants over time, but covering your lungs with gunk of any variety will decrease your oxygen absorption rate.

It's also worth mentioning that various brands of e-cig liquid have been tested by folks on the internet [citation needed, etc.] and all of them were found to contain trace amounts of nicotine. Not nearly as much as the normal "with nicotine" liquid, probably not enough to get addicted to unless you vape every time you breathe, and certainly not enough to do any statistically significant damage to you. But the stuff is made on the same equipment that makes the nicotine bearing product, and small amounts of it end up in there. If your goal is "none shall pass my lips" on the nicotine front, that's currently impossible.

Best bet? If you're not using them as a nicotine substitute to stay away from regular cigarettes (which are really bad for you) you may as well stay away from e-cigarettes entirely. They are not 100% risk free, so there's little point to using them. If you insist, the second-best bet is to treat them like cigars and not inhale fully.

The short answer is you're extremely unlikely to make any significant long-term impact on your health without something else -- Doritos, Mountain Dew, car exhaust, living in New Jersey, etc. -- killing you first.