OP should be able to establish any rule or boundary he wishes for his child and it should be respected. If he has lost respect and trust for his FIL, and tried to hash it out with him, at that point it's on the FIL to adapt.

If I can't trust someone, and they won't listen to me about what rules I make for my son, I'd have a really difficult time having them around him honestly.

It's not the FILs son, he doesn't get to dictate what's acceptable to do around OPs child.

I've had to come to terms with this kind of thing with my own family and it's incredibly draining and difficult to navigate but in the end this is the healthiest approach I've found to handle it.