There isn't a direct Rolex equivalent for my Grand Seiko (sbgx103) so I can't directly relate to your dilemma but one thing I do miss out on is attention.
For all the talk about how you should buy what you like, it's human nature to want validation and that others appreciate the choice that you made. I would say that this is particularly true for watches because high end watches are essentially functional jewellery (when any tourbillon-equipped automatic can be outperformed by a Christmas cracker plastic quartz then it's hard to argue otherwise) and so I want others to compliment me on my good taste.
But this has not happened - not once in nearly a year of ownership has anyone said anything nice about my watch.
The level of finishing is superb. Pictures don't do justice to zaratsu polishing, it has to be seen in the flesh to be appreciated. The blued second hand and GS logo seem almost alive as they subtly colour-shift in the light, and the dial itself is amazing too. I can (and have) lost myself for several minutes at a time just staring at this thing. Unfortunately, you have to see it close up to fully appreciate and from across a dinner table it just looks like a slightly-more-sparkly-than-normal conservative watch (an effect that could be cheaply replicated with a smattering of cubic zirconium).
It also commits the sin of being a quartz. Never mind that it's one of the finest quartz calibers ever produced, with finishing that shames most automatics. Never mind that Seiko specs this model to +/-5secs per year - and that it's lost less than a second since resetting for daylight saving in October. Never mind that it can go 50 years before servicing. Never mind the twin pulse motor with backlash control that stops the second hand dead on each marker. From the opposite seat of the train, enthusiast or not, it looks just like any other quartz mechanism.
For better or worse, you can spot a submariner/GMT ii/explorer ii from 6 feet away if you've got eyes like Mr Magoo. They are well known even to people who don't consider themselves enthusiasts so they attract the casual comments, and they are considered "proper" watches by enthusiasts who will then be happy to tell you what you want to hear - that you have a nice watch and they think you have good taste.
Tldr: no-one notices my watch. It makes me a little bit sad that I'm the only one to appreciate it.