The effect your describing is misleading. If my bank account gives me 6 cents per dollar I have in the bank in interest, you might say I get 6% interest. How often this is compounded(figured out and added) is important.
Its always 6% per year, but what if it's calculated annually? Dec31st roles around, my dollar is now 1.06. Gravy, right? Now let's do it monthly. After January I get my interest, 1/12th of 6% interest is .5% interest(remember I get this twelve times). .5% of my dollar is half a cent, now my bank has 1.005 in it.
This where things start to compound. That .005 attached at the end is now used in February's interest calculation, so I'm making interest on my interest! So on Feb 28th, I get to add my .5% interest and its .5% of 1.005 which is clearly going to be a bigger number than .5% of 1.00.
This is compounding interest and it has ramifications in tons of fields. What's important to note is that you can't just say "Well I want it make interest on my interest so fast its going to spike to a infinity". Think about what's added.
When we went from annually to monthly, our calculation dropped from 6% to .5% because I broke my interest up into 12 pieces. What happens if I do my calculation every day? My percentage daily and is going to plummet. Yes, it will always be a higher amount, but due to the interest rate being so incredibly small that after a certain point we plateau and by compounding every minute or second or millisecond we just can't seem to get much higher. Its always higher, just not by very much.
It just so happens we have a constant that describes the relationship between the plateau and the interest rate when compounding, as its an important number in mathematics and that's Euler's constant, also known as e. I'd heavily encourage you to go check the wiki article for this if your generally hateful towards letters in your math, its actually quite straight forward if you followed my explanation.