(Feel free to correct me if I'm wrong, I just glanced at a couple GIM Q&A streams/blog posts)
Prestige in a nutshell is just "we are the original members of the group formed from the beginning & no new member has joined us ever".
A group's prestige is lost when someone NEW joins a previously prestiged group (e.g. a 5-man group of OG members loses one person, and that group recruits someone new to fill out the gap).
I believe your individual prestige is tied to you staying within the group you originally formed, and it is lost when you leave your initial group to join another.
And I believe if you want to have group prestige, then you need to assemble your group on the Node before leaving for the mainland.