As a father of a 2yo girl myself, this hits me hard. I took a minute to search and here’s what I would do.

Go out and buy a hard drive. Make sure it’s an actual hard drive, NOT an SSD. 2.5” form factor is fine. Put the mp3 files of the recordings on it.

Flash storage, like a usb stick or an SSD or an SD card, can be subject to bit rot. Magnetic storage, like in a spinning hard drive, will be more resilient.

Make many copies, both on the hard drive itself, but also off-site. This can be cloud storage, another identical hard drive at a friend’s or relative’s house, etc. Get a box at a bank and stash one there.

Basically, as many copies on as many different hard dives, at as many different locations will help make sure these recordings aren’t lost, and as noted, the magnetic storage of the HDD will make it more resilient to degradation.

Then, buy any cheap MP3 player and just load these on. If the MP3 player gets lost, the file gets corrupted, etc, you always have a backup to restore from.