NTA but you also don’t know this dude’s life story. On average, Black people experience racism at a significantly high rate in ways that range from subtle to overt and non-white people historically have been documented to not understand it the way Black people do:

https://amp.theguardian.com/commentisfree/2018/jun/09/everyday-racism-america-black-white-spaces

I’m not sure what country you’re in but it’s okay not to lend out your phone. That’s not asshole behaviour.Questioning a Black person’s response to being denied a seemingly innocuous everyday service isn’t necessary. Just chalk it up to a slightly unpleasant situation but if it isn’t enough to get you to lend out your phone to strangers, push it out of your mind and move on.