I winced when I read this. I would like to propose that your father absolutely does know that he hurt your brother's feelings, and doesn't care. My father liked to get a reaction out of people, and to perform for an audience, so he would make condescending/belittling "jokes" to me and my brother all the time. and laugh off any negative reaction as us being hypersensitive or dumb. He was a bully and so is (probably) your father. That "offhand comment" was targeted specifically to wound your brother.

As far as your brother and father supposedly having a good relationship, if you hear negative crap like that all the time, and you can't easily escape it, as with a family member, you actually learn to screen it out to an extent. Your mother also sounds like an enabler, and there is a very healthy amount of denial going on on her part. Your brother is finally sick of your father's shit and of protecting your mother's feelings.