Maybe many will hate me but hear me out. Everybody deals with stress in various different ways. Throwing away a good marrige is not always as easy as many think, specially when there is a kid involved. I think this is all depending on the person and the relationship but I would highly suggest a couple therapy. I am not trying to justify or excuse him AT ALL but maybe, just maybe he needs help to get his shit together. Maybe he has some underlying issues that can be solved and you guys can have a healthy relationship. I think just forgiving and forgetting is the worst thing that can be done. The thrust is broken, he let you down on the most needed time. Awful everything. But share these concerns you have with him. Say that you are scared that he will cheat again, when stressed. You should both talk with a professional. It is just not normal to even be slightly horny, while your wife and your new born are about to die. He has problems, but just because he fucked up doesn't mean that you won't try to fix things for yourself and for him. You say that you are happy and things are good now. That is what a marriage is. Even through shitty times, trying to be better people and helping each other. At least to try. If it still doesn't work after the effort, then make your decision. It always seems the right choice to end it all and sometimes it really is. My advice is to try to understand it first. If you don't like what you understood or if he is not open enough you can always end everything.