I would argue from a comparative religion standpoint that the "decline of man from a Golden Age" is pretty universal, almost archetypical. Hindus, with no influence from Abrahamic religions, believe in such a thing, for example.

I think it usually has to do with the fall of some past order of things that's influencing the religions (The Destruction of the Temple, the decline of Rome and Stoic pessimism about human nature, the Buddhist retreat from a Hindu priestly class that was no longer up to par.)

For me, I tend to be rather more of a pessimist about human nature. Schopenhauer features prominently on my shelf of secular philosophy. I think it's kind of human nature to be jerks and greedy and whatever else you want to throw in there. We can make progress in external conditions but fixing human nature is still a long ways off. So in that sense, I can kind of identify with what the religious narrative tries to do- "we're all this way but look to these figures who gave to strangers and crossed deserts and so on"- the only problem is religion gives mixed messages on what's right and I don't think stories about "how to be good," even at their best, necessarily fix anything in practice.