I'm so sorry it's taken you this long! As far as being scared to go to the doctor and find out some thing is wrong, I can totally commiserate. I used to feel that way too, but honestly, once I went and found out I actually did have problems, it was a relief to know. Most female fertility issues are treatable or fixable. I was really far more upset at myself when the other shoe finally dropped and I'd realized I'd spent months running in a hamster wheel when I could have been starting medication and actually giving myself a chance to succeed.

And I do think that since TTC is such a goal oriented process, a lot of us really don't take the time we need to to make peace with the worst case scenario. It's a really amazing that we want a baby and that we're willing to do what we have to do to get there, but we gotta let ourselves make peace with the unlikely possibility that it might not happen too, if only to help ourselves out. Since I hit that wall and finally had accept at least the possibility, it's been much easier for me to be more proactive in my own health and care because I remember that doing so isn't just for a potential baby, it's for me and my long term well-being. This somewhat morbid lesson came to a head for me recently because once I finally started going to my OBGYN for fertility reasons, she discovered I had serious pre-cancerous spots on my cervix that needed to be removed right away.

In any case, I second what /u/UnicornToots is saying. Start temping. Call your doctor and make another appointment for yourself, and get a semen analysis done for DH. Better to find out now and get to work on fixing what you can then to find out later. It sounds like you guys are handling this super well and it's wonderful to have such a supportive relationship. That's awesome :)

Best of luck to you, I hope you'll keep us updated on your journey!