I was an altar boy for a few years. I preferred the later morning masses, but not the afternoon ones. Even did a few midnight masses here and there. You’re right though, church was always just the start to a Sunday. We’d go out to eat after, like anyone else, and then the afternoon almost felt more free than a Saturday. It’s like church forced us to not spend all day on some house chore project like I’ll often do on a Saturday.

The afternoon masses just mess up the flow of the day. I’ve already got something going, and it gets interrupted by church at 2 or 3. And it’s really interrupted before that because you’ve got to get ready for an hour, but for me it’s messed up even more because simply knowing I have something to do later in the day prevents me from starting anything, whether it be a chore or a hobby project. I end up just reading Reddit or watching tv, waiting for mass