NTA

If the cake was easily replaced and he ate it assuming there was more so he could replace the cake he ate before you ran out it'd be absolutely fine.
This however was a gift(and thus had sentimental value) that isn't replaceable. You don't do something irreversible to something with either of those properties without permission.

I suppose at a stretch he could argue he had implied permission because you let him have a slice the day before.
However not only is that a stretch it'd still be pretty rude to go taking another slice you'd just have reasonable grounds to argue you thought it'd be OK.
It's nowhere close to a defence to taking the majority of the cake, and an absolutely insult if someone thinks that means they can eat the full thing in a single sitting as he clearly did.