But there are tactics used in high optimized games that are unfun if you get targeted as a player: any save or suck mechanic, focus fire, wall of force and any AoE.

Part of high optimization is being able to handle when those tactics get thrown back at you. You've gotta be able to handle both offense and defense. If as a party you don't have a way to handle saves or suck, focus fire, or AoEs, then you aren't as optimized as you think. Maybe you'd have more fun if the table agreed to play the game at a lower level of optimization.

If your party doesn't know how to handle NPC spellcasters having silvery barbs, then the best solution is to agree for nobody on either side to use the spell.