No idea man, I'm going to blame your router because of the filtering it employs, and the way Bungie does matchmaking. You're breaking the methods Bungie are trying to pair you with players, and your router simply saying "nope, I'm not going to talk to them" isn't exactly doing the expected thing from Bungies perspective.
If players being matchmade with you are not connecting, then those clients will report an issue back to Bungie saying they can't connect to you, depending on how aggressive Bungies matchmaking is (Huh? he's working fine with everyone else, can you talk to them too? Yes? Well keep connecting then) would mean that they're being put in the session with you, regardless of what your router is doing.
The symptom your describing, redbars and yellowbars all the time is the game complaining of packet loss and latency issues, which makes sense if your router is filtering them.
Again, the unknown variable is the aggressiveness of Bungies matchmaking, and how it evaluates connection states in an inconsistent mesh (whereby the joining client can communicate with everyone else, except you). If Bungie is aggressive enough, that is, "I don't care if you can't talk to that guy, just join, I'm sure it's transient and will clear up, so you're in the session, just keep trying to connect to him" both your game, which knows it SHOULD be trying to connect to them (but your router is saying no) knows they're part of the session because you can't actually filter the matchmaking traffic, the actual communication your console has with Bungies servers.
Your console, and their console's are both saying "You're playing together" and there's this device in the middle (your router) saying "Nope, not gonna happen" ... and the code in Destiny isn't robust enough to understand this. The game uses UDP to communicate, which is stateless, unlike websites, etc which use TCP, and can detect state (SYN/ACK/FIN/RST packet loss, etc) Destiny doesn't go "Oh, you're not getting my packets? You better not play with me".
Another way of looking at it is Destiny is behaving in a bipolar fashion, Bungie is telling your console one thing, and it *should* be talking to the other consoles, but your router is filtering traffic, and absolutely nobody (except your router) knows it's doing it. So Destiny is going "Hello? Hello? Hello?, and nothing is coming back" yet matchmaking knows you should, so it keeps trying ... hence yellow/red bars because of packet loss (dropped connections).
I should note, very importantly, your router has to make an initial connection with the other person in order to evaluate the connection, it tests it over a period and then decides to either let the traffic continue, or drop it. It's during these initial bursts of traffic that the connection persists, and Bungie stops caring what happens afterwards (you drop them). Then when the physics host switches (something new I learned today) the whole process begins again, followed by odd behaviour.
That's the only explanation I can offer. Personally, if I was a family member, I would bring another router around to your house, plug it in, and sit there while you play a while. Unfortunately I tend to ignore people who play the "It's not me" card, I always, always test regardless of what the person says.
Did you reboot your computer? Yes? Oh well, we're going to do it anyway.
Did you reboot your router? Yes, Oh well we're going to do it anyway.
Did you turn off connection filtering? Yes? Oh well, let me in there, I will turn it off and test it anyway.