Not a Swede but living here for quite some time.

do the Swedish tend to delay parenthood

Yes. I think the average age for first time parents is rather close to 30. However, there is huge variation between the different regions. Parents tend to be younger in rural Sweden.

is it because they have better education/career opportunities and better social support? How is that impacting society?

There are a lot of reasons for this. For one, Swedes tend to want to "live the life" after upper secondary school and many have the means to do so. It is not uncommon to travel extensively (backpacking, working tourism, working in cafes/as guards to afford longer trips abroad). Many also study in Universities and the programs tend to be rather long (4-5 years). And since most people are about 19 when they finish upper secondary school that means that by the time you've got your University degree you'll be at least 24 (often older due to the travelling, trying out different things etc).

Also, the concept of relationships is a bit different here as compared to many other countries. First of all, sex education is obligatory and pretty extensive in schools. Also, there are free clinics for young people and it is not uncommon to get free condoms. Accidental pregnancies are there fore quite rare and so people don't end up marrying their high-school partner because one of them got pregnant. Second, it is a very secular country which means that its fine to be single, to date/sleep around and there isn't that much pressure to build a family in the first place.

Due to the possibility of education, travelling etc a lot of people think quite a bit whether or not to get kids. I get the impression that it is usually quite a informed decision. A lot of people feel the pressure to have their education done, have a stable job and permanent housing before they have kids. Which, of course is good, but in todays world also means that you'll be closer to thirty before you establish all these "requirements".