I agree that it's kind of silly if you lose it. The only time that's happening is if a high rank enemy targets it directly.

My bigger problem is that the spec is actually kind of... uh, boring? Your character becomes pitiful in this spec. No weapon swap, limited weapons to begin with, no tool belt, and the utility category being signets means you're going to use them passively much of the time.

It's basically a meme at this point that the golem is the main character and you're the pet, but this just makes it dull to play. Not to mention problematic for pvp modes, where the golem isn't even an issue. Players will just step around the golem and kill you instead because you're worthless.