You should definitely document it in a journal or something because if it escalates to the point of CPS coming, you can possibly lose the baby temporarily for letting the baby around that. I know it's not your intentions but do what you can to protect yourself and LO. If needed, get a temporary PFA against the Dad and his family if you're not living with them, so you can look into what you can do about the situation.

You can get a public defender to represent you in court to validate you having primary custody on paper so no one can try to take the baby away and you and the baby's dad can work out if you're going to do 50/50 or if he's going to get weekends or something. Definitely show proof of their drug use and you legally don't have to let them see the baby if you think they're doing suspicious activities. But on the other hand, since there's no court order in place, he can just take the baby and won't get into trouble, unless he's under the influence or putting baby in a dangerous situation.

Definitely look into the laws in your state. Contact an attorney and protect you and your baby legally. If the dad and his family don't want to respect you as the child's mother and threaten to take the baby from you, be armed and don't say a word to anyone what your doing. Again make sure to document everything.

This happened with my nephew. My brother got a free, public defender and got primary custody of him and they both had to do parenting classes, drug tests (she uses) and report to court. If you don't have transportation, there are buses you can take or sign up for free rides through the state for Dr visits, court, work, etc. The truth ALWAYS comes out eventually. I hope you get things worked out and can enjoy raising your LO in peace.