just because, it's weird
It's not. By making everyone white ( using your example) the Author is able to focus on different things in his story. The moment you add "diversity" you need to back it up.
If you want to create a consistent world, and care about world building in general, you need to justify the existence of a "non default" character(woman/black/gay/trans etc person). When your group of adventurers enters a village, you can't have everyone ignore the black person or the gay dude fucking the farm boys or the woman that should be in the kitchen making food.
That's what most people who have your position don't seem to grasp. This is a fantasy world, it is crafted, created, there's nothing random about it.
An author has a list of issues he wants to tackle, in that way diversity may be an asset ( for example malazan or wot, diversity in themes, not people).
The moment you add something non-tipical to your party, something that stands out ( in the world you created) you need to find it's place. If you suddenly have a black character (or any other race for that matter) you need a history. So, most people in this country are white, what is that guy doing here? how do the people react? is he a strange curiosity, considered a "lucky charm" to touch his big nose or just a demon? Same goes for everyone else.
This diversity thing hurts many novels, because most writers are superficial about it. They bring in present day politics into their novels to show how cool they are ( or to sell more novels). There's nothing more lazy than a medieval society having 21 century morals. It's even worse if everything is treated as normal, if this the village peasant is not surprised by the black dude who shows up by the inn, or the women wielding a weapon instead of working on the farm, or the gay ( the concept didn't really exist in olden times...at least not how we see it) dude casually hitting on someone and nobody taking offense.
TL;DR - We like to pretend being different is no big deal...and maybe it isn't in some countries, maybe in the future nobody will care( i mean honestly 100% don't care, to their core) what you are and how you behave, however if you want to add diversity to your novel you need 2 things: 1) make it "wrong", thereby adding conflict to your character as he deals with it. 2)Create a long backstory of this particular race/orientation/identity. your characters need to have opinions on this ( even if some of them don't care). If you don't want to explore this particular side of things then 1 ) make everyone the same. Everyone is straight and "white" ( human,elf,orc etc) there's no reason for conflict from this point of view.