You can't help who you are attracted to. That said, I've always wondered if people meant they didn't find certain facial features and body types attractive or if it was about melanin specifically.

When it comes to physical differences, we know that the factors there are complex. As far as I can tell, it's not inherently tied to race itself. We find a spread of physical features across all races, though we tend to think otherwise. To my understanding, there aren't even any specific racial genes.

http://physanth.org/about/position-statements/biological-aspects-race/

There are obvious physical differences between populations living in different geographic areas of the world. Some of these differences are strongly inherited and others, such as body size and shape, are strongly influenced by nutrition, way of life, and other aspects of the environment. Genetic differences between populations commonly consist of differences in the frequencies of all inherited traits, including those that are environmentally malleable.

Generally, the traits used to characterize a population are either independently inherited or show only varying degrees of association with one another within each population. Therefore, the combination of these traits in an individual very commonly deviates from the average combination in the population. This fact renders untenable the idea of discrete races made up chiefly of typical representatives.