As somebody who has played rugby and done a fair amount of combat sports over the years. You'd have to be hitting very very hard to do damage.

As somebody into risk reduction and safe play, this seems like awful advice.

Especially when you admit you don't do much impact play. There are unique risks with impact play, where, for example, sloppy technique can lead to wraparound which hits more sensitive areas with much more force than expected.

And using Rugby experience as your grounds for being a good judge of risk seems exceptionally flawed:

The study points to “conclusive evidence” that repeated impacts are linked to degenerative brain disease and says that those who play contact sport were at least 68 times more likely to develop chronic traumatic encephalopathy (CTE) than those who did not. (Source)