One thing that I don't see mentioned in the first dozen or so comments (at the moment anyway) is that speech doesn't require much bandwidth.

You can get surprisingly intelligible speech out of a very narrow frequency band. Humans can hear from about 20Hz to about 20kHz, but the voice band runs from about 300Hz to 3400Hz. And you can get intelligible speech with far less even than that - that range is actually fairly conservative.

The diminishing returns for intelligibility set in very quickly. There just isn't much point in a voice communication system having much higher quality - most of the time you don't even notice that the bandwidth is fairly low when you're understanding the other side just fine. The times it's really noticeable are when there's some background noise like music and you can suddenly tell that the quality isn't very high and it becomes harder to pick the speaker's voice out from the less-identifiable noise.

As a fun, unexpected (if you don't know about basic psychoacoustics) aside, the fundamental frequency of most adult voices is well below 300Hz, so you can't actually hear it over a typical phone. As it turns out, this isn't really a problem - you perceive the missing fundamental anyway as your auditory system automatically infers it from the harmonics.