Really depends on the person, and what you mean by "does it matter".

Like, can it be a dealbreaker? Yes. I had a relationship deteriorate due tous having an incompatibility with size. It wasn't the only problem but it was the largest one that contributed to the relationship ending.

I've had my size mostly cause problems in the bedroom with past relationships, it could cause my partners discomfort or pain if we were not being extremely careful, but my current relationship is very different. I'm able to give her consistent vaginal orgasms, which was something she had never experienced with past partners.

So the answer is essentially "it can matter in a variety of ways, but it depends on the person, and for most people it is a non-issue".