That actually makes it even harder to get to exceed the temperature of the source.

Ok, think of it this way. Suppose you had perfect mirrors literally all around the sun, focusing every point onto other mirrors, and all of those mirrors converged on a tiny point on the surface of the earth. In fact, suppose that tiny point is surrounded 360 degrees by mirrors. Now, you have perfect heating from the sun, there's no way for any heat to escape in any direction. What happens? The mirrors are two way, passive devices. The sun is a blackbody radiator - it is bright because it is hot. The point target very quickly reaches the same temperature as the sun. At this point, it starts emitting blackbody radiation at the same rate that it receives from the sun. Every single mirror also reflects perfectly in the other direction, transfering that energy back to the sun until the two are in equilibrium. The target can never be hotter than the sun.

Now, if your mirrors aren't perfect, that makes the target cooler, because the maximum temperature of the target is that of the sun. If the target isn't surrounded by mirrors, it starts to radiate energy to the earth and space, which are much, much colder than the sun. Thus the equilibrium temperature without perfect reflection will be cooler than the 5000 degree temperature of the sun.