My interpretation of his childhood would be how it's described in the Silver Eyes.

"Kind of petty tyrant, those who grew spiteful with their small scraps of power, feeling more and more abused by the year- by family who did not appreciate them, by neighbors who slighted them in imperceptible ways, by a world that left them, somehow, lacking something essential. Before (Carlton) stood someone who had spent much of his life fighting like a cornered rat that he had taken on the mantle of bitter sadism as an integral part of himself. He would strike out against others and revel in their pain, feeling righteously that the world owed him his cruel pleasures."