To put it briefly, disease leading to a decreased population, economic distress, rebellions, civil wars, misleading number of roman troops, and the actual size of the empire itself are all theories for Rome’s decline and difficulty in fighting outside threats. Due to the size and complexity of the answer required to fully explain this, I would like to focus on the military aspect.

By the time of Augustus, Roman expansion of territory exponentially decreased.  As a result, Roman troops and legions became increasingly tied to the frontier areas.  By the 2nd century A.D., temporary frontier forts built by the legions to provide shelter in winter became permanent settlements representative of small cities.  As a result, troops became increasingly tied to one particular region of the empire.  By the 4rd century A.D., new reforms took place trying to resolve this problem, splitting the army into 2 major segments, the frontier troops called the limitanei and the mobile army reserve called the comitantenses. The limitnanei main job was protecting the frontier borders from minor raids by barbarians numbering only a few hundred in most instances.  Unlike the limitanei, the comintansenses were designated as the mobile reserve, and designated to move any threatened area and remove the danger.  Unfortunately, the most important part of this reform, the comitansenses, failed in many respects, mainly the comitansenses became increasingly tied to a certain region or province, negating the purpose it was created.  Having the Roman troops isolated and tied to specific areas in the empire caused great harm in many respects, most importantly, it caused civil war.

Roman troop’s identity became connected to the local regions and its commanders.  When one emperor died or was seen as illegitimate, a commander with a significant army could declare himself emperor.  In other instances, the local military forces themselves would declare a new emperor.  This caused huge succession problems and led to an extremely dangerous pattern of civil war.  One saw many examples of this in the 3rd and 4rth century, the most famous example being Constantine whose Gaul based army militarily defeated the armies of the East in order to proclaim him emperor.  The civil wars played a huge part in weakening the Roman military, making it less effective in defending the empire from outside invasions.  

Economic distress, changes, and population decrease also weakened the Roman military.  For a large part of their history, the legions built and supplied their own weaponry and armor from military foundries. Essentially, if there was a need for equipment, the legion could supply that need themselves, effectively making them self-sufficient in that respect.  By the late Roman Empire, weapons and armor manufacturing had been given to nearby cities and professionals who built the equipment and supplied the local Roman troops.  This could be a huge liability especially when disease, war, or political corruption was in play.  Consequently, local roman troops could suffer from lack of supplies not representative of the legions from older eras.  Local political corruption is key here and leads me to my next point, the actual size of the Roman army and its potential for being a paper tiger.