You don't have to worry about it in tiers 1 and 2 play but AC definitely ramps up at higher levels which can bring down the effectiveness of animate objects a lot.
For example, recently my tier 4 artificer was fighting against a boss that had 20 AC. Before the fight I had the option of summoning either two of my eldritch cannons at the cost of two 1st level spell slots or casting animate objects using a 5th level spell slot. Here's how the damage of the two break down.
| Option | Attack Bonus | Hit Chance vs 20 AC | Full Damage | Effective Damage |
|---|---|---|---|---|
| eldritch cannons | +15 | 85% | 27 (6d8) | 22.95 |
| animate objects | +8 | 40% | 65 (10d4 + 40) | 26.00 |
If the battle lasted three rounds then using animate objects would have given a net gain of only ~9 damage (assuming I maintained concentration the whole time). In this case the clearly better option would be to summon the cannons and then use that 5th level spell slot during the encounter for some other damaging effect.