Sounds like a sore situation.
Perhaps he didn't know what he was getting into? Living with 4 other people might sound like fun at first but if I was at his place, I'd get very stressed out soon.
He might be aware of the talk that is going on about him (=others considering his behavior strange) and the pressure being put on him (=he should become less strange), and it's not probably making it any better. He might shut more and more inside his shell if he feels like he has permanently messed up the whole thing.
But it's not like you could do anything about it if he's not taking the initiative to meet you halfway. People will talk and there's nothing wrong in that - it's that when it's at the level when other people talk behind your back and think you don't notice, it's really hard to try to better the relationship with them. It feels fake.
It's clear you're the only person he feels comfortable around at the moment. Living with 4 other (previously unknown, except you?) people is really harsh for somebody who prefers one-to-one relationships. If he's anything like me, he doesn't care much about "hanging out together" if there's no meaningful relationship behind it. And usually it's not easy to share meaningful things about ourselves in a group. One-to-one relationships are better for that. He might need to get to know each one of you better before being able to be comfortable in group.
But yeah. Pretty bad situation. It sounds like he didn't know how things would turn out. He's probably not doing whatever he is doing out of being unfriendly - he's probably very insecure about the situation as well. Being aware of what is happening and still not being able to take an action doesn't help.. and I'm pretty sure he knows the weird atmosphere he carries around him and how others react to it. It's not nice for anybody.
Do you think there would be anything "easy" you guys could do together? Like watch a movie, cook food, play a board game? Something that would not put much pressure on anybody but that would make it feel more natural for you to be living under the same roof?
Or.. is there any possibility you could just stop caring, keep it casual-friendly and give him time? If the situation becomes unbearable, perhaps he will move out himself.