I've been in a similar situation too. I jut love being alone and having my space, to be left unbothered. For a very long time I've been telling myself I'm way too lonely and that i shohld try making closer friends, bring more people to my life.

But recently I've been talking quite a lot with this girl (I have zero romantic interest on her) and every time I find myself trying to have some excuse not to talk to her and do my stuff, since she demands attention and time (like any other relationship). Thus, I also feel selfish. But at least it helped me to realize I don't want to have many friends or talk to them everyday, I need one special person that I really like and likes me back, one that's worth my full attention and time (in a romantic way).

Answering the question, I don't think we're being selfish. Everyone is different and there's nothing wrong with liking to be alone and having your space but it still feels wrong, since, in my case, I'm doing something I don't really want and in a way I'm wasting her time