Do you own your own property? Do you have an HOA? Are your closest neighbors less than 300 feet away from your property line? If you said, "Yes" to the first & "no" to the second & third, feel free to destroy your house. Otherwise you're on the hook for damages.
Destroying a house in an area with an HOA lowers neighborhood property value, so you'll be in trouble with them. If your property is a mess, neighbors can have the city declare your house a public nuisance if there is the chance that rodents attracted by the mess run the risk of entering their properties & damaging them. Rats travel 100-300 feet from their nests when foraging for food.