Shannon Entropy of a Binary Source
Also known as binary entropy function · Shannon entropy · information entropy · entropy of a coin · bits per symbol · H(p)
Worked example: fair coin → exactly 1 bit. Press Try an example to run it live, then adjust anything.
Enter your known values, leave one input blank, and solves for the missing one. Tap a variable’s symbol to see what it means, with a typical value. Change any unit and the answer follows.
Where Shannon Entropy of a Binary Source fits
- Information and signals8 formulas
See also
Compression Ratio: entropy is the floor no lossless compressor can beatAverage Brute-Force Time: Both use entropy per symbol.Passphrase Entropy (Diceware and Word Lists): Both use entropy per symbol.Password Entropy (Uniform Random Generation): Both use entropy per symbol.Unicity Distance: Both use entropy per symbol.Shannon Entropy of a Binary Source explained
Entropy measures surprise. A fair coin gives bit per flip, the maximum, because you learn the most possible from the answer. A coin that lands heads 75% of the time gives only bits, and a coin that lands heads 99.9% of the time gives bits: you already knew what was going to happen, so being told adds almost nothing. At or the entropy is zero and there is nothing to transmit at all.
That gap between 0.811 bits of content and the 1 bit of storage a naive encoding would use is the entire business model of lossless compression. Shannon's source coding theorem says you can encode a source at its entropy and no lower, which means a 10/90 stream carrying bits per symbol can in principle be squeezed to under half its size, and cannot be squeezed further no matter how clever the algorithm. When a vendor claims 100:1 lossless compression on arbitrary data, this is the theorem they are violating.
Notice the shape of the function rather than any single value. It is symmetric about , so 0.25 and 0.75 give exactly the same entropy, and it is flat at the top. Moving from a 50/50 source to a 60/40 source costs you only 0.029 bits, about 3%. Most of the compressible slack lives out at the extremes, which is why real compressors spend their effort on long runs and rare symbols rather than on mildly biased ones. That symmetry also means the formula runs backwards to two answers: every entropy below 1 bit has two probabilities that produce it, and neither has a closed form, so the solver finds the one at or below one half by bisection and names its twin, 1 − p.
Shannon Entropy of a Binary Source formula
- = Entropy per symbol (bits)
- = Probability of one outcome
Cite this page
formula.expert. (2026, November 1). Shannon Entropy of a Binary Source Formula & Calculator. https://formula.expert/solvers/shannon-entropy-binary