There is such a thing as a second mortgage, but that's not what I'm talking about here.
Refinancing is basically taking out another loan to pay off the old one. This would typically be done to get a lower rate, but could also be done to extend the term of the loan (lowering payments), or to turn some of the equity in the home to cash (called a cash out refinance, where the new loan principle is higher than the balance on the old loan, because the borrower also received cash from the bank). When you refinance, you pay various fees, called closing costs, so refinancing can be profitable to the bank even before a single interest payment is received.
So a traditional mortgage has been a 30 year, fixed rate mortgage with 20% down. This means the interest rate stays the same over the life of the loan, and you put 20% down. Right now, the rate on this type of mortgage is around 4%, which is low by historical standards.
During the housing boom, other types of became popular. One type was the ARM, or adjustable rate mortgage. These would have an introductory rate, which may last as little as one year to up to 10 years. Then, the rate would vary based on an index rate (such as LIBOR) plus a margin. If the index rate goes up, so does your mortgage interest rate; if rates go down, so does your mortgage rate. So the borrow takes on some risk.
An ARM isn't necessarily bad; I have one, and now my rate is around 2.5%, which is fantastic (though I can't count on that to last). But some of these loans had extremely low "teaser rates" that would become much higher in only one or two years. The teaser rate might be 2%, then jump to 8% or higher. Because in the early years of a mortgage, almost all the payment goes towards interest, this means your payment would about quadruple. Because people used these teaser rate loans to get a more expensive home than they could otherwise afford, such an increase would be unaffordable. These loans were never meant to be kept. The intention would be to refinance every year or two, which is only possible if housing prices continue to rise or at least don't fall.
Another type was the balloon payment mortgage I mentioned earlier. Say you take out a $200,000 loan with an intro rate of 2% that lasts two years. At the end be of the two years, the entire balance comes due at once. The borrower would have to come up with almost $200,000 (because very little principle is paid off in the early years) immediately or go into default. Since most people don't have that sort of cash on hand (and those that do aren't likelu to take out this sort of loan), they have to refinance, paying off the old loan with the proceeds of a new loan. If you can't get a new loan because your home has gone down in value or you've lost your job, or if the new interest rate is much higher, you're in trouble.
And so is the bank, because they're no longer receiving payment. So both buyers and lenders were dependent on rising housing prices that enabled people to refinance. In retrospect, it was naïve to think this could go on forever, but a lot of people got caught up in it.