The short answer is that Russia is too big. If you look at historical hegemonies, the bigger powers will fend for themselves instead of subjecting themselves to a hegemony. "The West" here being an extension of American hegemony, America's sphere of influence and pertaining to American interests. The issue today isn't between liberalism and communism as it was during the cold war, but between liberal unipolarity spearheaded by America and multipolarity between the bigger powers in the world. Naturally, America favors the former and others the latter.

In other words, the cold war (as in the quest for influence) never truly ended because in terms of geopolitics it wasn't something unique to begin with despite being commonly conceived as an isolated and unique period in history. Today, if Russia and others were to let go of their immediate spheres and interests and concede to "The West", it would enable America to dismantle any SoI but its own, any geopolitical conception but its own and go for complete domination ushering in the unipolar liberal world order, which was already declared at the end of the cold war and described by people like Fukuyama. You might have heard people who champion this development be referred to as neocons.

You view "The West" as a monolithic thing which Russia has negated throughout history. This isn't the case according to me and I think Russians as well. They don't really view "The West" as this historical community, but more like an imperialistic concept and "natural sphere of influence" of America as described above. Not only that, but they view the countries that champion their "westernhood" as indoctrinated subjects of America. You may have heard of the term "Atlanticism" which seeks to compress the idea of The West into a less mysterious historical concept and more of a American geopolitical conception. Other such "natural spheres of influences" could be Panslavism, "Central Europe" (German empires), Communism, "Orthodoxy", Shia (Iran), etc. All powers tend to have one an idea like this.

Now, the broader and more abstract notion of "The West" is a relatively recent idea, beginning sometime between the Renaissance and Enlightenment through its reverence and romantization of classical antiquity. It laid the foundation for a common Western/European identity and culture. Note that this reverence itself is the foundation, not actual classical antiquity. If you actually look at the ancient world, you'll find that Greece and to a lesser extent Rome probably had more in common with places like Egypt, Tunis, Anatolia and Syria than the barbarian hinterland in Europe which makes up The West today and were contemporary western ideas originated. Greece is cited as an originator of the west and thus the first western country, but historically, this doesn't really make sense. It's like calling China a "western country" based on newfound reverence for Chinese culture and philosophy in Europe two millennia later, when obviously China had more in common with its Asian neighborhood, just like Greece was more of an "Eastern Mediterranean" civilization. Rome on the other hand, whatever its own foundations, directly projected into Europe and laid a genuine foundation for "European culture". So to use the China analogy, imagine if China colonized Europe and brought its ideas.

You could perhaps even trace the idea of the West back even further to the rift created in the Mediterranean world by Christianity, as well as the rift between Catholicism and Orthodoxy, but I think the Renaissance and Enlightenment encapsulates the effect of those events already so I won't comment on it.

Westernhood became a thing once liberalism and capitalism truly took hold and we move away from the aristocratic and feudal structures which governed most of the world - including Europe. If you ask me, the historical "European values" are actually aristocracy, consolidation and imperialism, not democracy and humanism as we like to pretend today. Thus before liberalism, Russia wasn't any less "western" than the rest and primarily engaged in politics with Europe on similar terms and with similar attitudes.

So what changed? We have the introduction of the nationstate, republican/liberal democracy, secularism, free economy, industrialization, etc. Adoption of these concepts, even outside of geographical Europe, would render you "western": Israel, Ataturk, Nasser, Japan, etc. These ideas are still fundamental to "westernhood" today.

Empires like the Ottoman Empire, Austrian Empire and Russian Empire became outdated and plagued with various combinations of the concepts mentioned above. Other polities like Britain and Sweden found a compromise between new and old. During this brief period, I would deem Russia outdated and thus un-western. The Russian Empire might have had a hard time eschewing ideas of the Renaissance and Enlightenment, but so did many other European powers, and if you ask me I'd blame proximity and the natural unwillingness of the ruling class before some unique flaw in the Russian character.

But after the revolution, what exactly wasn't "western" about Russia? Russia didn't take the liberal route but went straight for communism which too descended from the Enlightenment. The Russian Empire was split up into federal nationstates. Surely, this must make them the most progressive and western nation of the day. If Japan was westernized then communist Russia and China were westernized tenfold.

But no, they still weren't western. Despite whatever changes in politics, technology and industry, and so on, they will always be poor and brute, the people are controlled by an intellectual elite, completely detached from ruling class ideas no matter how western they might be. But is that really the case? If it is, were they any special? Is it true today? At the end of the day, it seems like it boils down to stereotypes mixed with geopolitics more than anything else.

[Excuse the messy post, I was writing it in segments. I also welcome any disagreements.]