I admit I'm the person who jokingly comments "You're so quiet!". It's never meant to make someone uncomfortable but when I say something like that, it's an invitation for you to "contribute" to the conversation or let me know if I need to back off, shut up, keep talking, or whatever depending on your response. Saying things like:
"Oh, I just like listening" tells me you're comfy and okay, maybe even having fun.
"I'm usually just really quiet" or "I'm kinda shy" tells me you're not a talker, cool. I can work with that.
"Yeah, so?" tells me our personalities probably won't mix so I should leave you alone.
"I get told that a lot." comes off as passive aggressive and I have no idea what that means.
"I don't know anything about this topic so I can't really chip in." tells me that the conversation needs steered somewhere you feel included again.

We're not trying to call you out as being flawed, we're just trying to be overly considerate of everyone in the conversation. It's awkward as hell to be talking and someone doesn't chip in to the conversation. Like, are you just tired? Do you need a hug? Do I need to bebop right up out of here and leave you alone because you're secretly thinking of smothering me with the nearest hoodie just to shut me up? Did I offend you and somehow missed a signal? We ask because we care, that's the short and sweet of it.