Shannon Entropy.

A fair binary event carries more uncertainty than one that almost always repeats.

Interactive experimentintuitiveField note ·

Which coin is harder to predict?

A fair binary event carries more uncertainty than one that almost always repeats.

PLAY & COMPARE

Chance of heads50%

1%99%

PLAY THE IDEAMake a choice

The next toss has a 50% chance of heads.

Choose a move to see what happens.

Binary entropy

Entropy is the expected surprise before one binary observation.

Entropy describes uncertainty in a model, not meaning or usefulness of the message.

Preparing the experiment…

THE IDEA

Entropy measures average surprise under a specified probability model.

For a binary event with probability p, H(p)=−p log₂ p−(1−p) log₂(1−p) bits. Entropy peaks at one bit for p=0.5 and approaches zero at the extremes.

Estimate how much information a yes/no answer may carry when one answer is rare.

WHERE IT BREAKS

Entropy describes uncertainty in a model, not meaning or usefulness of the message.