Two things not considered here:

• You aren't using gibberish. Your password is going to be a word, like "coat" or "dogs." There are significantly less four letter words in the English language than there are four letter combinations.

• Oftentimes, the attacker already has access to an encrypted form of your password, such as a leaked database. At this point, there is nothing the original website can do to throttle the attacker. They are brute-forcing your password on their own device, no Internet connectivity needed. As the top comment says, it would take 1 minute per account to crack a 4 character password. It would take 3.5 days to crack a longer one.

When a database gets leaked -- even if the passwords are encrypted -- you'll be glad you were forced to make it 8 characters with symbols.