Round lenses are easy to make and mount and are a bit more durable since they don't have corners to act as stress concentrations.

Like sure you could make a square lens if you really really wanted to, but it would look like you punched a rectangle out of a round lens, meaning the edges would be different widths. Technically it isn't necessary, glasses uses rectangular lenses after all, it's just easier.

Add on that lots of applications like a zoom lens can require things to rotate.

As for why pictures are square, that's mostly about how we view and organize things. Square (rectangular) things are easier to divide into smaller square things. Square things stack neatly, etc. Having a round display, or photo, or otherwise feels like you are missing space at the corners just based on what we are used to working with.