I forgave him and pretty much let it go-except once in a while I get mad again and make comments about it.
You either have to let it go or not. Keeping him around to have him berated occasionally over something he's apologized for and stopped is a recipe for disaster.
I’m just feel a little alone and unwanted and not sure what to do
Is there any chance he's feeling the same?
I take excellent care of my kids, clean the house, laundry, and cook all the meals
You didn't mention any way in which you try to take care of your husband emotionally. Any chance he's feeling neglected with all your focus (understandably) being on the kids?