Aspergers doesnt have anything to do with 'not too smart' except for maybe when it comes to social and emotional intelligence and reciprocity, but those can be issues for anyone. Maybe he just doesnt give a shit and its a nice guy, maybe dim, and its just his personality.

Whether its some sort of disorder or not shouldnt really matter to you though. These are clinical labels intended to guide clinical legal and occupational interventions and family and self education which are applied when there is clinically significant deficits and dysfunction in social emotional and occupational functioning. Theyre not useful in the way you are trying to apply them. People are way more complex than that. Everyone deals with these issues as a part of growing up.

I wouldnt try to diagnose people. Just let them be who they are and decide whether you want to be friends with them or not. Dont be so judgy. Its pretty obvious from your post and comments that you have deficits in social functioning. Just focus on that.