That's precisely why those countries are tax havens in the first place. At least relative to profits made in the US, companies can only use a country as a tax haven by engaging in an accounting trick called "transfer pricing" - which is also illegal. But the only way to prove that transfer pricing is going on is to obtain records from the companies that they hide in the tax haven jurisdiction.

As far as how the US government stops other countries from doing that - its hard. The US managed to get Ireland to promise to sort of stop allowing companies to do it there by 2020, but only because the Senate came very close to passing economic sanctions on Ireland.

Most other popular tax havens, such as the Cayman Islands, are nominally part of the UK which makes dealing with them much more difficult from a political perspective.

With profits made within the EU its a little bit more difficult because the EU has provisions that explicitly allow for companies to shift profits from one EU member state to another.