Early on, never wasn't saluting someone up or down. Jr. officers have it the worst, obviously. Once upper field grade, returned a lot and still initiated a lot when in higher staff positions. That's how it works.
Unfortunately, some ossifers lose respect for basic customs & courtesies. Sets a poor example, as they disdain saluting as a chore.
It's a necessary, outward show of respect for every uniformed service member, regardless of rank. There's plenty of appropriate no hat/no salute places and situation when deemed most appropriate to the circumstances.
Tough shit, I say. Respect yourself and customs & courtesies, regardless of rank. I also tired to keep a sense of humor when purposely caught with hands full, etc.