Maybe it has a certain amount of RAM and crashes itself by growing too fast, or runs out of storage capacity.

Go out on the internet, and take over a million PCs. Plenty of people are already doing that right now, using no more tools than a basic internet connection and their brains. And they are not always getting blocked quickly, especially if they aren't causing immediate trouble.

Don't even think about countering with "well the AI would just hack it ways in". No, that's not how it works.

That's how it works right now for well equipped intelligence agencies. Complex systems always have bugs. Look at spectre/meltdown for instance.