There is no win-win here.
You have a deep obligation to your Brother; but he also has a deep obligation to you and to your House - and he is repeatedly violating his obligation by transitioning to a woman in a male-only organization - and by (arguably) sexually harassing current Brothers. In my view, he needs help the House cannot provide - help to transition if that is what he really wants - and help to understand his transition is not permission to act out, sexually harass, or knowingly damage the recruiting of his own House as a political act of misplaced inclusiveness.
Your Brother needs to be separated from the House and told that if he comes onto House property, a formal restraining order will be sought from a judge - an order that will appear on criminal background searches and damage his ability to secure meaningful employment or decent rental housing for years to come. At the same time, Actives who are comfortable with him should continue to visit him and do things with him off-House-grounds - so that he understands he is not being abandoned, even if he is no longer welcome on grounds or as an Active.
Following my advice will absolutely trigger blowback from him, from his instigating friend, and from the trans supporters on campus - that blowback may actually be the intent of his actions, instigated by his activist friend. There will be marches and demands and perhaps even vandalism. You may need to hire security; you definitely need outside cameras surveilling your grounds.
The guys best suited to Rush, Pledge, and Initiate - will not be comfortable with a sexually aggressive transsexual living, sleeping, and showering among them. If you want your House to grow, separating your transitioning Brother is necessary - and it can be reliably done by pointing out that your Brother is now a woman (according to his own activist group's definitions) - and this is not a space where women live, nor an organization where women are members. You cannot be true to your mission to burnish, better, and Bond the best guys coming to campus if you also admit women.