I've mostly compared the Dreamcast to the PS2 and have generally found that the Dreamcast's 3D output results in a far smoother picture than the PS2, ignoring resolution differences. Compare Soulcalibur on the Dreamcast with the aliased edges in Soulcalibur II on the PS2, for example. Sharp transitions from one colour to another introduce dot crawl artefacts in composite video and so the Dreamcast's smoother picture will help somewhat here. The smoother picture will also look better over VGA/YPbPr when blown up to fill a modern HD panel as well, huge jaggies are never particularly appealing!

The article PS2: Aliased No More is quite interesting, though it does go on to say that it's not just a lack of antialiasing that's the problem with the PS2.