Entropy Analysis Guide: Shannon Entropy, Password Strength, and File Security

Entropy measures how unpredictable or random a sequence of symbols is. It shows up across software development in ways that matter: judging password quality, detecting encrypted or compressed files, testing random number generators, and identifying packed malware. Use the Entropy Calculator to follow along with the examples below.

The Shannon Entropy Formula

Claude Shannon introduced the formula in his 1948 paper A Mathematical Theory of Communication. For a sequence of symbols drawn from an alphabet, entropy H in bits per symbol is:

H = -Σ p(xᵢ) × log₂ p(xᵢ)

Where p(xᵢ) is the probability of symbol xᵢ appearing. The result is measured in bits per symbol. Entropy is maximized when all symbols are equally likely (uniform distribution) and zero when only one symbol ever appears.

The maximum possible entropy for an alphabet of size N is log₂(N) bits. For a byte (256 possible values) that ceiling is 8 bits/byte. For a single ASCII printable character (95 values) it is approximately 6.57 bits.

Worked Examples

Notice that "AAAABBBB" and "ABABABAB" have identical symbol frequencies and therefore the same entropy (1 bit/symbol) — Shannon entropy captures symbol distribution, not order or structure. This is an important limitation covered at the end of this article.

English Text Entropy

Measured character-by-character, English text sits around 4–4.5 bits per character (26-letter alphabet, non-uniform distribution). But Shannon's own experiments, accounting for word and phrase-level redundancy, put the true entropy of English closer to 1 to 1.5 bits per character.

That redundancy is precisely what compression algorithms exploit. A typical English plain-text file compresses to roughly 25–35% of its original size with gzip because most character combinations are highly predictable given context. High-entropy data — already random or already compressed — does not compress further.

File Entropy in Security

The byte-level entropy of a file is a reliable heuristic for its contents:

Security tools use a sliding-window entropy scan across a binary to flag high-entropy regions. A normal Windows executable has low-entropy code sections (repetitive opcodes) and a small high-entropy resource section. A packed executable looks almost uniformly high-entropy from start to finish — a classic indicator of UPX, MPRESS, or custom packers used to evade signature-based detection. YARA rules can encode this check directly.

Password Entropy vs Password Strength

The entropy of a randomly generated password is straightforward: a password chosen uniformly from a space of N possibilities has log₂(N) bits of entropy. For a 12-character password using lowercase letters (26) that is log₂(26¹²) ≈ 56.4 bits. Using all 95 printable ASCII characters it rises to log₂(95¹²) ≈ 78.9 bits.

Entropy is an upper bound, not a guarantee. A human-chosen password like Password123! occupies a 12-character, mixed-case, digit, symbol space (nominally high entropy) but appears near the top of every credential-stuffing list. Dictionary attacks and rule-based mangling reduce the effective search space from billions to thousands. The entropy formula assumes the attacker does not know your selection strategy — when the strategy is predictable (keyboard walks, leetspeak, appended years), entropy overstates strength by orders of magnitude.

Related: Password Strength Explained and Password Generator Guide.

Base64 / Hex / Raw Binary Entropy

Encoding affects measured entropy because it changes the alphabet and symbol distribution:

• Raw binary — maximum 8 bits/byte when fully random.
• Hex encoding — uses only 16 symbols (0–9, a–f), so maximum entropy is log₂(16) = 4 bits/symbol. The same random data encoded as hex reads as ~4 bits/char.
• Base64 encoding — 64 symbols, maximum log₂(64) = 6 bits/symbol. Random bytes encoded as base64 approach ~6 bits/char.

If you scan a binary and find a section whose base64 per-character entropy is suspiciously low (well below 5.5), the underlying data is likely structured or repeated. If it is at the 6-bit ceiling, it is almost certainly encoded random or encrypted data.

Tools for Entropy Analysis

• ent (ENT Pseudorandom Number Sequence Test) — classic Unix CLI that reports entropy, chi-square, arithmetic mean, and serial correlation for binary files.
• Python scipy.stats.entropy — accepts a frequency array and returns the Shannon entropy in nats (base-e) or bits (base-2).
• YARA rules — the built-in math.entropy() function lets you write rules that match on high-entropy sections inside a file offset range.
• Shannon Entropy Calculator — browser-based, works on pasted text or uploaded files, shows per-symbol breakdown.

Code Examples

Shannon entropy in JavaScript:

function shannonEntropy(str) {
  const freq = {};
  for (const ch of str) freq[ch] = (freq[ch] ?? 0) + 1;
  const len = str.length;
  return Object.values(freq).reduce((h, count) => {
    const p = count / len;
    return h - p * Math.log2(p);
  }, 0);
}

console.log(shannonEntropy("AAAA"));   // 0
console.log(shannonEntropy("ABAB"));   // 1
console.log(shannonEntropy("ABCD"));   // 2

Python with scipy:

from scipy.stats import entropy
import numpy as np

def file_entropy(path: str) -> float:
    data = open(path, "rb").read()
    counts = np.bincount(list(data), minlength=256)
    probs = counts / counts.sum()
    # base=2 gives bits; default is nats
    return entropy(probs, base=2)

print(file_entropy("sample.exe"))   # e.g. 6.34

Sliding-window entropy for file analysis:

def sliding_entropy(data: bytes, window: int = 256) -> list[float]:
    results = []
    for i in range(0, len(data) - window, window):
        chunk = data[i : i + window]
        counts = [0] * 256
        for b in chunk:
            counts[b] += 1
        h = 0.0
        for c in counts:
            if c:
                p = c / window
                h -= p * (p.bit_length() - 1)  # fast log2 approx
        results.append((i, h))
    return results

Use Cases

• Password generator QA — verify that generated passwords approach the theoretical maximum entropy for the chosen character set.
• Detecting encrypted or compressed data — entropy near 8 bits/byte means the data is already opaque; further compression or encryption adds no benefit.
• Crypto key randomness testing — combine Shannon entropy with a chi-square test to confirm that a key or IV source has no detectable bias.
• Packer / obfuscator detection — flag executables or scripts where overall file entropy exceeds 7.2 bits/byte as candidates for deeper inspection.

Pitfalls

• Small sample variance — entropy estimates from fewer than a few hundred symbols are noisy. A 10-byte sample can look high-entropy just by chance.
• Shannon ignores order and structure — "AAAABBBB" and "ABABABAB" have identical per-symbol entropy. A sequence with strong positional patterns (like a structured header) can still score high if its symbol distribution is flat. Use autocorrelation or compression ratio alongside entropy for a fuller picture.
• Language-specific character sets — comparing entropy across texts in different languages or encodings (UTF-8 vs UTF-16 vs Latin-1) without normalizing the alphabet produces misleading numbers. Always measure entropy over the same token unit (byte, code point, character) consistently.

Measure entropy on any text, binary, or password with the Entropy Calculator — runs entirely in your browser with no data leaving your machine.