I wouldn't characterize them as xenophobic so much as supremacist, both biologically and culturally. There's a very real element of social Darwinism in Klingon philosophy and society, where victory over enemies, strength, and such are seen as being the same thing as inherently better and more honorable. This cultural mindset lends itself to supremacy when Klingons meet cultures which do not conflate victory with worth, because whenever a human or a Vulcan or a Romulan decides to take a route other than direct, often violent confrontation, they're seen through the lens of Klingon thought to thus be weak.
Martok, I think, represents two major ideas within Klingon society:
Like Worf, Martok is very much a traditionalist, a Klingon conservative who seems to value the older ways. While he is very much interested in glorious combat for personal honor and for the Empire, right away you get the impression that he's not interested in superfluous or needless battle, particularly against weaker opponents. Martok is from the old school of Klingons, who understood that honor and combat were not interchangeable, who understood that not all instances of violence were appropriate, and who recognized that honor can take many more forms. When Nog is trying to prove himself as a security officer on Depp Space Nine, Martok recognized honorable bravery even in an unexpected package and gave Nog his due respect. When Worf was fighting the Jem'Hadar one after the other on the prison asteroid, after a certain point Martok felt that Worf had more than proven himself a great warrior.
Speaking to a degree from experience, there can be a certain necessary gung-ho culture among those of lower ranks in the military, one which worships machismo and dehumanizes the enemy as a defense mechanism against allowing basic humanity to prevent one from doing one's job. If your job is to capture and take a civilian village in which there are insurgents hiding, sympathy can lead to hesitation. Speaking purely from a military viewpoint and dismissing the ethical viewpoint, too much humanity can be dangerous. There needs to be a certain allowance for black and white thinking, for dehumanizing others, for allowing one's ego to make them feel powerful. This kind of thinking is useful for troops on the ground, but for higher up in the military hierarchy it can become a liability. Those who lead have to be willing and able to think about the battlefield and the context in which the battlefield exists in shades of gray. If you're just an enlisted Klingon with bat'leth in hand about to go into combat, you need to think of yourself as the hero and your enemy as the weaker villains who need to be destroyed. Martok, however, needs to think about the bigger picture.
[please note that the above is intended to be analysis within a fictional universe and speaks in generalizations which could be construed to but which are not intended to speak disrespectfully of soldiers]
Because Martok is both a traditionalist and is required as a general to think in far more subtle terms than supremacy, he tends to hold Starfleet in higher regard than some fellow Klingons.