In nearly every single game I enter, there is someone much higher level with extremely good stuff. I've came to realize this only happens when you pay money though.

Really depends. I've spent zero and could've been 40+ if I really wanted to thanks to luck and time played. Money spent makes it easier to progress and gives you more chances for good things, but RNG is RNG.

Will there ever be a way for players who dont pay to be able to obtain this stuff without having to get EXTREMELY lucky?

No, but there isn't really a way for people who pay to get that stuff without being extremely lucky either. The easiest way to understand this is to remember that every 100 V-Bucks you get is like $1 spent, so if you got the believer pack and all your free daily ones, that is like $22 spent and you walked away with nothing of note.

Likewise, I have 15 legendary items, zero cash spent and was not someone who bought an Troll llama. This isn't a brag per se, just a reminder that RNG is going to determine what you get and money spent only increases the number of rolls you get.

I mean id love to stop using this blue quality soldier after 100 hours of play you'd think I could at least have a purple?

Again, RNG. You could spend $10,000 and still not have a purple or better. At that point you can use the legendary transformation key and get one, but that isn't really the point. The point is, like it always is, is that you can't buy good or bad luck. It just happens. All paying does is increases your chances or guarantees things like the Troll loot.