I understand what you mean. I have a number of thoughts, idk if any will be helpful to you but here goes:

1.) You identify the problem right there: your perception shapes your reality. Your worry affects how you respond to things and interpret them. So this is an issue of managing your worry, rather than acting in a particular "right" way for any particular situation.

2.) People are wrapped up w/themselves/their own thing. So much so, that even if whatever you're doing seems a little unusual, it will likely be forgotten about. Or rationalized away, like "That stranger's walking so quickly because they're late for a meeting."

3.) I calm myself by simply/kindly narrating what is happening. How would a reasonable, understanding person explain what's occurring? "I happen to be within hearing distance of these strangers. Because of that proximity, some times I can't help but take in some of what they're saying." Okay, that's understandable. This situation is just a little inherently awkward, that's all.

4.) A basic suggestion I have is to find the purpose, in those anxiety-driven moments. What is your purpose in that moment? Is it walking around aimlessly for enjoyment? Is it travelling for groceries? Deciding what to do with your time? Find your purpose for any given moment and return to it. Focus on that instead of all the potential interpretations of how people may misguidedly interpret what you're doing.