Seems like you have not used a proper time-out. His room is probably filled with toys. Use a boring location.

When he has a negative interaction, put him in time-out immediately and don't talk to him or look at him while you are putting him in time-out. If you much say something, then use a calm voice, don't look at him and make is short like "no pushing". The time-out should not be longer than 4 minutes and 1 minute might be long enough.

You also might try removing her again, but with special care to the details of how you do it: When the unwanted behavior starts, immediately pick her up and take her away without looking at him or talking to him. If he follows, ignore him, not talking to him or looking at him for up to 4 minutes.

Give him high quality positive attention for any good or merely acceptable interactions with her. Do it immediately, Get close, use affectionate touch, be as enthusiastic as he can stand, say specifically what you say (not just "good job"), don't caboose criticism on the end (no "but...").

In other words, get control of your behavior-getting attention and direct it toward the behavior you want and away from the behavior you don't want.

Using this kindness chart might help you and your spouse start focusing your attention on the opposite behavior so that you will reinforce it a get more of it:

https://familylinks.org.uk/sg-cms/pdf_docs/KindnessCharts.pdf

Talking though it and Screaming at him are attention and parental attention is the single most powerful positive reinforcement at this age. The habit has formed because you have reinforced it with attention. If you and your spouse make a consistent change, then the habit will go away in a week or two.

This is based on 50+ years of parenting research and parent training results:

http://epicurusgarden.blogspot.com/2013/08/the-reinforcing-power-of-adult.html