I had a very poor relationship with my father ever since I was a teenager, as far as I can tell I was just a disappointment compared to my brother and my father disagreed with how I lived my life. When my wife and I announced our first pregnancy he just changed the subject. Didn't even say congratulations. This would have been his first grandchild. I say "would have" because he died before my son was born. After he passed, I heard from family and his friends how much he was looking forward to become a grandparent.

Some people are just weird and don't deserve your love, doesn't matter if someone is related by blood if they don't act like family. I saw my father once a year around christmas, and even that had gotten too much. I understand how different the situation is when you have children, especially if your child loves your father, but I would cease to make any effort and more or less cut that toxic shit out of your life.