I'm getting things together for my next graphic novel, which will have a painterly look. Here's some things I've been experimenting with:
Art brushes. Use the ones that come with Illustrator, make your own, download some off the net. Don't limit yourself to ones designed to imitate real media - some of my most interesting effects have come from using a set of abstract "tech shapes" brushes.
The Appearance palette. Using Roughen is a good start - but play with the parameters, stack some other effects on top of it! Try effect->roughen (2 pt, absolute, 15/in) followed by effect->tweak (10/10pt absolute), for instance. And then maybe drag those effects onto the fill, duplicate the fill, set the new one to like 50% and tweak its effects a little. Maybe make the new one size up a little, play around with what's available as live effects!
Scatter brushes. Draw an abstracted little blob of mess, randomly scatter it tightly around the edge of a shape. Maybe even stack up a couple different scatter brushes with different in the Appearance palette. (Hey, that's a pretty good idea, I should try it... wow, this looks pretty good after tweaking the brushes so as to have distinctly different repeat periods.)
Bristle brushes. You can do a lot with them. And you can apply effects to them - try taking one of the stock bristle brushes, then click on the stroke in the appearance palette, add 'outline stroke', add 'roughen', then drag 'roughen' just below 'outline stroke' for a really beautiful, soft, messy brush.
Once you make an interesting appearance stack, save it as a graphic style, so you can quickly select it again. Some of these things can be a real pain to recreate from scratch.
WARNING: stacking up lots of stuff in the appearance palette can result in Illustrator having to create a TON of points, which can bring it to its knees. You will get pauses and crashes; SAVE OFTEN. Ultimately you have to find a happy medium between beautiful messes and being able to actually use Illustrator without having to constantly wait for it to draw the preview window.
Hope this helps, good luck!