Here's the summation:
Manziel was expected to challenge Brian Hoyer, but (according to media outlets) did not actually show enough progress to become the heir apparent to the Browns...yet. Coaches felt he was limited on the field and felt off-the-field, he wasn't learning plays and wasn't giving his all, too committed to his own ego and lifestyle.
When Hoyer started to struggle, local media, the front office, and a good subsect of fans wanted Manziel to come in and Kaepernick the situation; save the Browns' playoff chances with an incredible run, but he hadn't had enough time to learn, and (supposedly) Shanahan HATED him and sabotaged the play calling. Manziel was inserted into a struggling team that was fighting for its life with a banged up offensive line, divisiveness between the coaches and front office (supposedly), and not enough preparation to face top-flight opponents.
The Bengals saw this and ate Manziel alive. The Panthers opted to do the same and knocked Cleveland right out of any small hope for contention. They exposed the weaknesses that many had been worrying about with Manziel for years. The front office, in the process, alienated Hoyer and pissed off Shanahan and Dowell Loggains to the point of no return.
Maybe Manziel would have worked out if he sat for the year, but we all (the team, the coaches, the FO, the fans, the media) got impatient and threw him to the wolves too early. He needs an offense built around him, and the Browns are still working to make that happen. But currently, they can't even gauge it; Manziel is still in rehab and they've lost Brian Hoyer, the hometown guy.
It wasn't a complete and utter disaster, it was the summation of a lot of disasters that have let the media at-large and the fans to have a field day on how the Browns have been trying to build this team. The real judgment will come this season, as, barring a miracle, it's likely Manziel will get a 16-game shot to prove he's changed and can lead the franchise. And if he fails, well...Cardale Jones calls in 2016.