Okay, so I just found out what a permanent is and here’s what Wikipedia says about how it differs from the determinant:

The definition of the permanent of A differs from that of the determinant of A in that the signatures of the permutations are not taken into account.

So I believe the best way to understand it in a physical sense is;

Assuming how the ‘determinant’ is the measure of volume scaling factor of the linear transformation produced by a matrix where the sign shows whether the transformation preserves or reverses orientation. So, since the signatures of the permutation are not accounted for in the ‘permanent’; it is just the measure of the volume scaling factor with no details about the orientation of the transformation.