Espionage is useless unless you invest very heavily into it, which means dedicating extra envoys, influence, and bonuses to Codebreaking to it. And spending all of those resources on espionage comes at a significant opportunity cost, since they're all more useful being spent on other things. Like, maybe if one of your friends went Psionic (+2 Codebreaking/Encryption as part of the Psionic tradition) and was willing to heavily invest into espionage for the sake of your federation, while the other two of you cover other areas of the game, the smear campaign thing could have worked.

The only thing you can do is just be larger/more efficient than 7 AI empires with 3 player empires. You could level up your federation, coordinate your expansion and military action (three humans should be smarter about coordinating than the AI), release sectors as vassals and specialize them, specialize your own economies and then help each other out with trade deals and diplomatic agreements: e.g. one player takes Masterful Crafters and uses Civilian Economy, the other takes Relentless Industrialists and Militarized Economy, CG specialist trades excess CG to the other players, Alloy specialist trades excess Alloys to the other members, the third goes all out on Research and signs Research Agreements with the other two, things like that. Just one example. If you all chose different climate preferences, you could also sign migration treaties with each other so that you can colonize all planets within your borders with high habitability species. There's lots of things human players can do when coordinating with each other that are way beyond the capacity of the AI, especially if you really coordinate your empire builds before the game starts and really coordinate your plans once the game starts. By doing that, choosing different origins, different specialties, different ascension paths, you can share all your strengths with each other.