Honestly your husband needs to grow a spine and go to court and have a custody schedule put onto paper. “Open door” very rarely works long term because inevitably someone gets jealous, or kids lie about things like you’re experiencing, or someone disagrees with the parenting style in your house and they think they have a right to say anything.

If he won’t, then you need to take your house to family counseling. His children acting like that is unacceptable. Has anyone asked them point blank why they’re lying? Not in an accusatory way but, “hey, I heard you told your mom X. We both know that didn’t happen and it was extremely painful to me to hear this. What’s happening?”