Can't say for sure. Using magic to control another character is PvP, but if your character was doing something that goes against the interests of the party, that's also PvP.

If you can visualise it as a tug of war between two or more players, it's PvP.

Whether it's justified depends on whether you're trying to break or restore the balance.

So... you be the judge. If this was entirely unjustifiable, you should probably have an out-of-character chat before the next session. In-character, if someone constantly mind-controlled you with magic without any justification, you would have no reason to want to travel with them. Realistically their character would, at best, be kicked out of the party and at worst, they'd be be beaten half to death and left at the side of the road. I mean, who the hell wants to travel with someone who would use such invasive magic on their allies?