I agree, I think from the first scene with the talk show, we already get some evidence that he likes to daydream about being liked/loved. Then when he goes to the comedy club, he bombs and then we transition to silence so we can't hear the joke but apparently it's good enough to warrant sudden applause? Then the girlfriend who seems to show up at the most random times, where Arthur is suddenly quick-witted and normal and confident and smooth?

I would have much preferred that they removed the flashbacks just so that the audience could piece together that clearly those girlfriend scenes were also in his head given her response, "Your name's Arthur, right? You live down the hall?" Like clearly they haven't actually been hanging out.