CM · Noise and information

Channel Capacity and Shannon

However clever your code, there is a ceiling on the bit rate a channel can carry without error. Learn Shannon’s conclusion that the ceiling is set by only two things — bandwidth and SNR — and how to trade bandwidth against power.

Can this channel carry the target rate?

Sweep the SNR. When the capacity curve C/B = log₂(1 + SNR) rises above the dashed target rate, that rate can be sent without error (gold); while it stays below, no code can do it (red).

Signal-to-noise ratio SNRSNR = 3.0 dB

Sweep the SNR. Capacity rises only as the log of SNR.

Capacity versus target at this SNR

C/B = 1.58 < 4

Below capacity

Can cleverness be infinitely fast?

If noise flips bits, you can send each bit several times or add a clever code to cut the errors. So if the code is intricate enough, can any noisy channel be made as fast and as accurate as you like? Shannon’s 1948 answer was startling. No — there is a hard ceiling. But below that ceiling the error can be driven as close to zero as you wish.

Shannon’s formula

The value of the ceiling is written in one line: C = B log₂(1 + SNR). Here C is the maximum bits per second sendable without error, B the bandwidth, SNR the signal-to-noise ratio. Neither the modulation scheme nor the kind of code appears in it. A channel’s capacity is set by only two numbers, bandwidth and SNR. If the rate Rb is below C, a suitable code can send it arbitrarily reliably; if Rb is above C, no method can avoid errors.

ObserveC = B log₂(1 + SNR)

Capacity is set by bandwidth and SNR alone.

ChooseRb ? C

Only below C can you send arbitrarily reliably.

The trade — bandwidth and power

There are two ways to raise capacity: widen the bandwidth B, or increase the signal power to raise SNR. But the two act differently. B sits in front of the formula linearly, so doubling it doubles capacity; SNR sits inside the logarithm, so gaining one more bit takes roughly doubling SNR. Power gives diminishing returns fast, bandwidth keeps paying. So when power is scarce you widen the band, and when band is scarce you pour in power. Even with unlimited bandwidth, a floor on the energy per bit remains that cannot be beaten.

Fill inη = C / ? = log₂(1 + SNR)

The efficiency per hertz is log₂(1 + SNR).

On your ownB → 2B : C → ?

Doubling the bandwidth doubles capacity (SNR fixed).

Back to the first screen

At low SNR the capacity curve lay below the target rate, and that rate was red — sendable by no code. As you raised the SNR the curve climbed slowly, crossed the target, and turned gold. Because the curve is a logarithm, each further step up cost ever more SNR. The one thing to hear in channel capacity is this: the absolute ceiling on error-free communication is pinned by bandwidth and SNR alone at C = B log₂(1 + SNR) — everything below it is possible, everything above it is impossible.

The channel capacity C = B log₂(1 + SNR) is the maximum bit rate a channel can carry with arbitrarily small error. Independent of modulation or code, it is set by only the bandwidth B and the SNR. A rate below C can be sent reliably with a suitable code; a rate above C cannot. To raise capacity you increase bandwidth (linear) or power/SNR (logarithmic), and the two can be traded.

In the next unit

Shannon promised the error can be driven near zero below C, but not how. The last unit is the how: error coding. Add extra bits to the message by a fixed rule, and even when noise flips a few, the receiver uses those extras to notice the discrepancy and fix it. We start from a single parity bit and see how errors are detected and corrected.