As inspiration, have you seen: http://longnow.org/clock/ ?

A big problem, you will face with electronics, will be that "wet" components (like electrolitic capacitors) will fail relatively quickly (decades).

Simple semiconductors (LEDs, transisters ect.) and dry components (resisters, inductors ect.) have huge predicted lifespans.

Unfortunately PV cells do deteriorate over time, whether this stabilised over time, you could check. Maybe Copper oxide PV cells would be more stable. IDK.

In the type of timeframe you are talking about, even things like aluminium heatsinks might be out, due to corrosion.

Maybe look at something like a joule thief driven LED cast in a clear block of plastic. Could even harvest: light, heat differentials and RF energy... just don't use wet components and over spec everthing.

I would like to see what you come up with