Thank you for the answer! I like your response to the physics example (which is one of the most common responses), although it probably wouldn't convince the person who came up with it, since he believes in some sort of asymmetry between the goodness of pleasure and the badness of pain.

Good point about exponential discounting for uncertainty. You're right that it's hard to determine which way the EU calculation goes. If the utility grows faster than the discount rate, doesn't that make the EU calculation a divergent series with infinite expected utility? (Or astronomically large but bounded by the heat death of the universe.) Can we rule that possibility out?

What do you expect will happen in the next few decades that will give us better ways of estimating probabilities, that we do not already have now? Are you anticipating advances in epistemological theories, improved technology and computer programs, or something else? Creating a program to accurately estimate real-life probabilities, like E.T. Jaynes's Bayesian robot if you're familiar with his book, seems like an AI-complete problem to me. You probably mean something less sophisticated than a full oracle AI, though.

By the way, what books would you recommend on formal epistemology, social epistemology, and your other areas of interest? I have already read Martin Peterson's intro to normative decision theory and Kahneman's work on cognitive biases and heuristics.