There a number of things that make this question difficult to answer. I'll provide some points.
Firstly, very few people have experience at multiple universities. Not only that, but very few people have experience as a student, at the same stage of development, at different universities.
Asking general questions about 'is it hard' depends on discipline. Geosciences will be different from Physics and different from History and so forth.
Workload is accompanied by expectations of work completion. For example, at Oxford, missing a college tutorial is a very, very bad thing. In fact, at many colleges, missing multiple tutorials will likely result in disciplinary action. There is the expectation, for instance, that you live on campus and do not have a part-time job.
In addition to the above, many degrees work via 100% final examinations. So even though you hand in work, the work does not count to your final grade. Despite this, Oxbridge students almost always submit their coursework. At other universities, many students will not submit non-graded coursework.
So you see, my point is that comparing workload at University A to Oxford is difficult because the typical student at University A may not be subjected to the same expectations (despite the fact work isn't counted). Oxford students are highly micromanaged, so even the lazy ones are subjected to constant reminders by their tutors to study.
The standards are higher at Oxford. That's just a function of higher expectations and the students.
Is it unbearable? Not really. In one year, something like 4-5k students graduate from Oxford. If it was unbearable, then that wouldn't be the case.
In general, I'd say that a high 2:1 at Oxford is largely equivalent to a 1st at most other universities in the Top 15-20. And vice versa. That gives you an idea of grade equivalence.
The other big change from Oxford/Cambridge to other universities is the spread of abilities. The spread of abilities at a university like, say, UCL is extremely large, with a much higher number of people who should really not be at university doing their chosen degree. The average talent is higher---probably consistent with the high 2:1 to lower 1st I noted.
I should say that many top-tier talent at other universities would be 1st class students at Oxford.