My point is that we don't see any actions firsthand to justify thinking of him as this ultimate evil

I understand you just fine. I just think you're mistaken.

Even if you limit this to just The Lord of the Rings, this isn't true. It's stated plainly in the text that Sauron is a malignant force directing and controlling (almost to the point of being a puppeteer) his forces when they march out to destroy the heroes in the story. The only way you get around that in the text is to not take Tolkien literally, which is counter to his stated intentions.

And there are first-hand accounts of characters in the story interacting with him, like the Elves who knew what Sauron was doing (personally) when he crafted the Ring and first put it on, or Elrond who was at the battle at the end of the Second Age when Sauron personally lead his forces to battle to destroy the armies of the Last Alliance. Centuries are pretty meaningless when the characters involved are immortal or nearly so.

He's also never referred to as an "ultimate evil." He's just the most evil thing in the book. And there are no redeeming qualities ascribed to him anymore.

It's not like Sauron is not a name the characters put to the concept of evil in the story. It's an actual person who rules over a country that's evidently determined to destroy the countries where the main characters come from.

And again, the entire point of Sauron is that his type of evil is one of domination, and the evil he's committed is demonstrated in the twisting of the world and its people. All sorts of evils are committed directly because of Sauron's orders, plans, and will enacted through his servants and sometimes by himself back when he still had a body.