expenses for this wedding have used up almost all of our savings

You accepted an invitation you couldn't afford. If it's taking all of you and your wife's savings to be at your brother's wedding, you should have sent a card and a gift and stayed home.

while possibly letting him know just how much of a burden he's placed on us? Am I being selfish by just thinking of me and my bank account?

Yes, you are being selfish. You don't accept an invitation to a destination and wedding and then complain about the expense. And your brother didn't place the burden on you -- you chose to incur the expense when you made those reservations. You brought this up once in a semi-joking way, and it backfired on you. Don't bring it up again.

One course of action here is to try to find another person who could take over your travel and lodging reservations, if that's even possible, and reimburse you.

If that's not possible, see if you can get refunded in credits with the airlines and hotel. At least that way you and your wife can have a holiday together sometime in the future.

If the only way forward is to attend the event, then do it with a pleasant smile on your face. You fake it every step of the way, because this is not your brother's fault.

You shouldn't have accepted the invitation, but since you did, participate pleasantly and make sure your misplaced anger doesn't leak out to anyone.

No moaning and complaining to your brother or his fiancee, or other guests or family members. Be pleasant to everyone.