Britain, Germany and France all saw massive pogroms against the Jews;

The only pogroms in Britain happened during the middle ages, such as the York massacre where ~100 were killed. The Jewish population of Britain at that time was tiny (around 5,000) and the entire community was expelled in 1290, so saying there were "massive" pogroms is an overstatement. Likewise in France, aside from expulsions I don't believe there were large scale pogroms there. Most of the pogroms in pre-Nazi Germany occurred between the 16th to 18th centuries, when Jews were forced to live in segregated ghettos (e.g the Frankfurt ghetto where the Rothschilds were from) under very harsh conditions. The largest pogroms occurred, naturally, where the vast majority of Jews lived historically, that being eastern Europe.

In the Middle Ages they did not share the Christians' religious limitations on banking, so they got quite rich.

In most of western Europe, Jews were barred from usury after the 13th century. Additionally, the Catholic laws against Christians practising usury were relaxed around the 15th century, which is why you see Christian bankers such as the Medicis of Italy and the Fuggers of Germany flourishing during this period. After the expulsions of the middle ages, most western Jews migrated to Eastern Europe, and the majority of them were very poor. Jewish banking didn't begin to recover until the 19th century, and so Jews involved in banking today are wholly disconnected from the Jewish moneylenders of the medieval period.

this explains why there's so many Jews in the upper classes, politics and entertainment.

Actually, there were very few Jews considered as "upper-class" until recently. Prior to the 20th century, around 75% of the world's Jews lived in a place called the Pale of Settlement, an area that encompassed large swathes of Poland, Lithuania, Belarus, Ukraine and Western Russia. It was created by Catherine the Great in 1791 as a sort of ethnic enclave (perhaps most analogous, ironically, to the Palestinian West Bank), where Russia's Jews were to be confined to one region. Under the Russian Tsars, life for Jews of the Pale was extremely harsh and poverty-stricken. Besides curtailing their freedom of movement, a quota system was put in place that either restricted or completely abolished their participation in education, professional occupations and voting, amongst numerous other disabilities.

Meanwhile, Jews in the West faired little better. they were almost invariably segregated into urban ghettos under the harshest of conditions. They were almost universally denied the right to vote, enter politics, gain higher education (by consequence excluding them from law and medical professions) or even in some places to marry without a permit. As late as the 19th century, Jews were still denied access to craft and trade guilds, thereby severely limiting their employment opportunities.

To use the Ghetto of Rome as an example:

“The ghetto was a walled quarter with three gates that were locked at night. The area of Rome chosen for the ghetto was one of the most undesirable quarters of the city, subject to constant flooding by the Tiber River.

Life in the Roman Ghetto was one of crushing poverty, due to the severe restrictions placed upon the occupations that Jews were allowed to perform. Roman Jews were allowed to work only at unskilled jobs, such as ragmen, secondhand dealers or fish mongers. They were permitted to be pawnbrokers (which had been prohibited to Christians); and this activity excited the hatred of many Christians against them. When Jews went outside the ghetto, the men had to wear a yellow cloth (the “sciamanno”), and the women a yellow veil.

The great number of people living in such a small area, together with the poverty of the population, caused terrible hygienic conditions. During the plague of 1656, 800 of the 4,000 inhabitants died because of the epidemic.

The Roman Ghetto existed from 1555 until the late 19th century.”

Jewish social disabilities barely improved throughout Europe until the 19th century. Beginning with France in 1798, countries across Europe began abolishing the ghettos and affording civil rights to Jews. This period is often referred to as the “Emancipation of the Jews”.

Beginning in the late 19th century, following the pogroms in Russia in 1881, there was a great migration of Jews to the west, particularly to America. Being a relatively new immigrant group, with the vast majority arriving after 1900, Jews from the beginning faced tremendous social discrimination in America. Being newcomers meant that they were naturally looked down upon by the establishment in almost all walks of life, and aside from ethnic and religious discrimination Jews also faced significant economic hardship to begin with. Most Jews back then were very poor, the vast majority having migrated from Eastern Europe, most notably the Pale of Settlement. In 1924, the Johnson-Reed Immigration Act was introduced – mainly to restrict the number of Jews entering the United States. Unemployment was rife in the early days as many businesses refused to employ Jews, whilst even some hotels and restaurants had a "no Jews allowed" policy.

Exclusive country clubs and golf resorts would routinely deny Jews entry. To give one notorious example from the 1940’s, when Groucho Marx was prevented from using the swimming pool of a Los Angeles country club for being Jewish he famously quipped “My daughter’s only half Jewish, can she wade in up to her knees?”.

Most Ivy League colleges at one time had ethnic quotas which limited the number of Jewish students, with some prestigious private schools excluding them altogether. Many middle and upper-class neighbourhoods had restrictive covenants which barred Jews from moving into the neighbourhood. Much of this lasted well into the 1980's, despite anti-prejudicial laws coming into effect during the '60s as a result of the civil rights movement.

TL;DR: Jews have historically been persecuted mostly for religious and ethnic reasons, not because of economic envy. Until relatively recently, the majority of Jews were poor working class people with limited civil rights.