If they call the police every time you shoot, but that isn't enough to qualify as harassment, what if you shot more often?

If you speak to a lawyer I'd specifically ask about your rights to tell the police that you aren't interested in talking to them anymore. It may be reasonable to tell them that no, you don't have to chat with them every time the neighbor hears you plinking. You might also consider No Trespassing signs. These make it easier to arrest the neighbor for trespassing if they get aggressive. A fence and locked gate could also be a good idea, depending on what your road frontage looks like. This could limit the circumstances when police could just pull into your driveway and up to your house. They will have specific rules about when they can bypass physical barriers.

I want to preface this last thing by saying that they should absolutely be coming to you as neighbors if this or something like it is the case, and asking to work out a mutually acceptable solution. Do you know why they're so anti-gunfire? I have vet friends who struggle with fireworks and distant gunfire. And friends whose rescue dogs who freak the hell out at loud noises.