Since a large part of my empire "was" pacifist I lost a ton of unity
That sounds like a problem with your empire management. Faction unity production is fairly minimal even with modifier bonus like oligarchy, shared burdens etc. Likewise any approval loss from faction unhappiness should be easy to mitigate by raising living standards or increasing amenities.
declaring a war. Even tho it was to end a threat.
Why are you even declaring war?
From your post it sounds like you joined one of the awakened empires as a vassal to fight the other one. That is a terrible idea.
If you're within 20k fleet power of one of the awakened empires you should always remain neutral or join the League of Non-aligned powers. Allowing subjugation, even a temporary one under a awakened empire is bad.
If you remained neutral or joined the League you could have just waited until one of the awakened empires declared war on you and avoid most of the opinion penalties from the pacifist faction since, that would be a defensive war. Furthermore there is absolutely no doubt that at least one of the awakened empires will declare war eventually, their AI is ridiculously aggressive. If they are taking too long just make a claim on one of their territories. I guarantee you'll provoke the war declaration within the next few days.
I've been slowly moving my pops to mostly xenophobic
Why xenophobic? Xenophobia is not the opposite of pacifism, militarism is. Read how the ethic change mechanic works for fanatic ethics: https://stellaris.paradoxwikis.com/Ethics#Changing_ethics
Additionally xenophobic ethics doesn't offer much late game. Who cares about Starbase cost and pop speed growth gained so late? And attempting to shift your empire to accommodate slavery, the main appeal of xenophobia, can cripple your economy for very little benefit. In the late game most, if not all your pops should be working ruler or specialist jobs.
Militarism on the other hand offers strong bonuses to military power and access to very powerful civics like Distinguished Admirality.