However you feel about figuratively meaning literally now, is the answer to the question.
Language is so completely arbitrary that semantic debates are void of any inherent meaning or purpose. Argue semantics and you're essentially two sides both punching at hallucinations. In the end there was no value gained or lost, just some punched air cause when you fight made up stuff you do nothing of any real value.
So you're in a debate with zero ground rules, or anchors over qualifying to meet a words qualification, but the qualifications are so subjective they ain't mean shit other than what you want it to mean.
What your left with is, the winner is decided by which side has the most people, not through semantics, debate or logic. So whatever side has the most people decides what the word means, does it involve physical exertion or not?
That's all there is to it. Two sides fighting over made up shit with zero basis to base an argument on. So pick your imaginary side, hope your side has more people, or are just much louder, and someone says "alright, I guess sport means X now".
When ya consider how much English, as a whole, has changed over the millennium, you may ask yourself, "who cares?" I mean, someone speaking Ye Olde Days English from like 800AD or English's birth time, while Shakespeare is sometimes hard to follow due to differences in the evolution of English from then to now, its at least readable and understandable ( resembling enough of today's English you can make enough sense of the outdated language parts to at least have a clue) while the OG early English, before even Shakespeare general time period changed shit, like happened between then and now, wouldn't make one lick of sense to any English speaker today. You'd have like zero clue what it means, likes its a complete different language.
Tl;Dr: there is no right or wrong answer, so just pick a side cause ya might as well be discussing space whales when the subject at hand is 100% nothing but arbitrary anyway. Space whales are at least entertaining.