He said to be in before 9 AM if it's possible (I had to sit down with HR for a brief meeting and she asked me if I had any issues)-- that's all.

I think your last point hit the spot-- I'm not too annoyed by him asking me to come in earlier, but he's never explicitly said it (it's always the other people's opinion), while he comes in past 11 AM every day unless there's a meeting at 10 AM. This sets up a weird time dynamic between him and I as well, since I come in a full three hours earlier than he does.

I think in the end the problems I had (I deleted my post b/c I think I got the answers I wanted from this thread) was that he wasn't being explicit: "I want you in before 9 AM every day because that's what you should be doing" vs. "I think you should come in before 9 AM because so and so might have an issue with you coming after 9 AM every day" are two different notions in my head, and that he's not making an effort to come in earlier since I am to better help my progress and be available when I am working to ask him questions (which I tend to have a lot of even as I work independently-- nowadays I just go to other people who are on my team that are in earlier)