The typical male reaction to hearing about a problem is furiously thinking how he can fix it himself. In your case this can be mildly conflicting for him because 1) He doesn't want you to change a whole lot (he's with you after all) 2) While you see the core of the problem to be your own shortcomings, to him the core of the problem seems to be that it's affecting you in a negative way.

So you both see the same thing but may be diverging in how you interpret it. For men who are very masculine, hearing complaining might alarm them because, well, complaining can only be the result of dire straits. This is because people tend to project their own attitudes and features on to others to varying degrees, even when they're very much conscious that people are different and vary in character.

Like others I suggest dealing with these issues internally. You can utilize journaling to make a clear picture of said past events and come to terms with what took place, including who was at fault, but I suggest forgiving yourself if you are to blame, because it's not exactly easy to make up for what took place in the past, especially when you were a minor.