I googled "applications of reinforcement learning" and found many interesting things.

You would not use RL for classification/regression since those are not "dynamic" problems where the environment changes with actions of the agent.

Recommender systems/search could be viewed as a reinforcement learning problem if you interact with the user and can refine your recommendations over time.

Deep learning in general, and RL with deep learning, is very data hungry. That is why games are great since you can generate an infinite amount of data. For robotics, Google had this farm of many robots all trying to grasp objects which was meant as a way to generate data for RL. Using many robots makes it faster than using a single one. But now people are using simulation for robotics and figuring out ways to have what is learned in simulation be transferable to real life.

Otherwise, the lack of data is a very big problem for deep learning, especially in RL. AlphaGo must have generated millions of games to become an expert. You couldn't train a robot surgeon on millions of patients before it gets good enough. There is a lot of research on deep learning to make it more data efficient; it would also be closer to "real" intelligence.