CM · Noise and information

Channel Capacity and Shannon

However cleverly a code is designed, the bit rate a channel can carry without error has a ceiling. This unit covers Shannon's conclusion that the ceiling is set by just two things, the bandwidth and the SNR, and how bandwidth and power can be traded against each other.

Can this channel carry the target rate?

The curve plots the capacity C/B = log₂(1 + SNR) against the SNR, and the shaded area beneath it is the region that can actually be reached. Raise the SNR with the slider and the vertical line marking the present point slides to the right; when the curve rises above the dashed target rate it shows gold, and while it stays below it shows red. Staying below the target means no code at all can send at that rate.

Signal-to-noise ratio SNRSNR = 3.0 dB

The target rate is 4 bits per 1 hertz, which the curve reaches only once the SNR passes 15 (about 11.8 dB). Because the curve is logarithmic, raising the SNR lifts the capacity only slowly.

Capacity versus target at this SNR

C/B = 1.58 < 4

Capacity too low

Can cleverness be infinitely fast?

If noise flips bits, sending the same bit several times or adding a clever code can cut the errors. Then, given a sufficiently elaborate code, could any noisy channel carry data as fast and as accurately as desired? In 1948 Shannon's answer was a surprise: no, there is a ceiling that cannot be crossed. Below that ceiling, though, the error rate can be pushed as close to 0 as desired.

Shannon’s formula

The value of the ceiling fits on one line: C = B log₂(1 + SNR). Here C is the largest number of bits per 1 second that can be sent without error, B is the bandwidth, and SNR is the signal-to-noise ratio. The formula contains neither the modulation scheme nor the type of code, so channel capacity is fixed by just two numbers, the bandwidth and the SNR. If the intended rate Rb is below C, a suitable code can send it as reliably as desired; if it is above C, no method can avoid errors.

ObserveC = B log₂(1 + SNR)

Capacity is set by bandwidth and SNR alone.

ChooseRb ? C

Only rates below C can be sent as reliably as desired.

The trade — bandwidth and power

There are two ways to raise capacity: widen the bandwidth, or boost the signal power to raise the SNR. Their effects differ sharply. With the SNR held fixed, bandwidth multiplies the whole formula, so doubling it doubles the capacity, but the SNR sits inside the logarithm, so gaining 1 more bit takes roughly doubling the SNR. Adding power quickly runs into diminishing returns, which is why a power-starved link widens its band and a band-starved link pours in power. Yet if the band alone is widened at fixed power, noise grows in proportion to the band and the SNR falls, so however wide the band becomes, the energy needed per bit never drops below a fixed minimum.

Fill inη = C / ? = log₂(1 + SNR)

The efficiency per 1 hertz is log₂(1 + SNR).

On your ownB → 2B : C → ?

Doubling the bandwidth doubles capacity (SNR fixed).

Back to the first screen

At a low SNR the capacity curve lay below the target rate, a red zone where no code could send at that rate. Raising the SNR lifted the curve slowly past the target and turned it gold, and because the curve is logarithmic, each step up demanded an ever larger SNR. One thing to remember: the absolute ceiling for error-free communication is C = B log₂(1 + SNR), fixed by bandwidth and SNR alone; everything below it is possible and everything above it is not.

Channel capacity C = B log₂(1 + SNR) is the largest bit rate a channel can carry while keeping errors as small as desired. It depends not on modulation or coding but only on the bandwidth B and the SNR. Below C a suitable code can send reliably; above C it is impossible. Capacity rises with bandwidth (linearly) or with power/SNR (logarithmically), and the two can be traded against each other.

In the next unit

Shannon promised that below C the errors could be made almost 0, but he did not say how. The final unit takes up that method, error coding. Attach extra bits to the data according to a rule, and even if noise flips a few, the receiver can use the extras to notice the mismatch and correct it. The unit starts from a single parity bit and works up to the principles of detecting and correcting errors.