Kullback-Leibler Divergence D_KL(P || Q), Cross-Entropy & Jensen-Shannon Calculator

Compute H(P), Cross-Entropy H(P,Q), D_KL(P||Q), and symmetric Jensen-Shannon Divergence.

Forward KL Divergence D_KL(P || Q)
0.184 bits
Cross-Entropy H(P, Q) in Bits 1.065 bits
Reverse KL Divergence D_KL(Q || P) 0.194 bits
Symmetric Jensen-Shannon Divergence 0.047 bits (0–1)
Inputs
⚡ Scenarios:
P(1)
True probability P(1) of outcome 1 (P(0) = 1 − P(1)).
Q(1)
Approximating probability Q(1) of outcome 1 (Q(0) = 1 − Q(1)).
Results & Graph
Forward KL Divergence D_KL(P || Q)
0.184 bits
Cross-Entropy H(P, Q) in Bits
1.065 bits
Reverse KL Divergence D_KL(Q || P)
0.194 bits
Symmetric Jensen-Shannon Divergence
0.047 bits (0–1)

Forward KL Divergence D_KL(P || Q)

Graph 1 • KL & Jensen-Shannon Divergence • Domain Architecture & Live Telemetry Maps True Distribution P1, Model Distribution Q1 through KL & Jensen-Shannon Divergence Formula.
Forward KL Divergence D_KL(P |: 0.184 bits KL & Jensen-Shannon Divergence • Invariants & Metrics Forward KL Divergence D_KL(P 0.184 bits Cross-Entropy H(P, Q) in Bit 1.065 bits Reverse KL Divergence D_KL(Q 0.194 bits Symmetric Jensen-Shannon Div 0.047 bits (0–1)
Graph 2 • Forward KL Divergence D_KL(P || Q) Across True Distribution P1 Sweep (40% → 160%) Evaluates D_KL(P || Q) = Σ P(i) × log₂(P(i) / Q(i)); JSD = 0.5·D_KL(P || M) + 0. across 7 operating points (current input highlighted).
0 bits 0.627 bits 1.25 bits 0.0880 0.280P(1) 0.00300 0.420P(1) 0.0350 0.560P(1) 0.184 0.700P(1) 0.471 0.840P(1) 1.00 0.980P(1) 1.14 1.12P(1)
Graph 3 • Joint Sensitivity: True Distribution P1 × Model Distribution Q1 Compares Forward KL Divergence D_KL(P || Q) when Model Distribution Q1 is shifted +25% (solid) vs. −25% (dashed).
0 bits 0.574 bits 1.15 bits --- Model Distribution Q1 @ 75% 0.0150 -30% 0.00300 -15% 0.0580 +0% 0.189 +15% 0.426 +30%
Summary: Shannon entropy H(P) is 0.8813 bits, Cross-Entropy H(P,Q) is 1.0652 bits, D_KL(P||Q) is 0.1839 bits, and symmetric Jensen-Shannon Divergence is 0.0467 bits.
Summary: Shannon entropy H(P) is 0.8813 bits, Cross-Entropy H(P,Q) is 1.0652 bits, D_KL(P||Q) is 0.1839 bits, and symmetric Jensen-Shannon Divergence is 0.0467 bits.
⚡ Send result to: Scientific →Statistics & SD →Probability & P-Value →
Interactive chart & math engine powered by CalculatorGraph

Formula & Mathematical Methodology: KL & Jensen-Shannon Divergence Formula

DKL(P || Q) = Σ P(i) × log₂(P(i) / Q(i))
JSD = 0.5·DKL(P || M) + 0.5·DKL(Q || M), M = (P+Q)/2
D_KL(P || Q) = Σ P(i) × log₂(P(i) / Q(i)); JSD = 0.5·D_KL(P || M) + 0.5·D_KL(Q || M), M = (P+Q)/2
  • trueDistributionP1: True Distribution P1 (P(1))
  • modelDistributionQ1: Model Distribution Q1 (Q(1))

Rounding: Internal floating-point calculations run in 64-bit IEEE-754 double precision; display values round to 2–4 decimal places.

Worked Example: Worked Mathematical Example: KL & Jensen-Shannon Divergence

Compute H(P), Cross-Entropy H(P,Q), D_KL(P||Q), and symmetric Jensen-Shannon Divergence.

  1. Specify the function parameters, matrix entries, or confusion matrix counts.
  2. Apply the closed-form or iterative formula: D_KL(P || Q) = Σ P(i) × log₂(P(i) / Q(i)); JSD = 0.5·D_KL(P || M) + 0.5·D_KL(Q || M), M = (P+Q)/2.
  3. Examine the primary result, error bounds, and 3 interactive SVG plots.
Result: Computes KL & Jensen-Shannon Divergence via D_KL(P || Q) = Σ P(i) × log₂(P(i) / Q(i)); JSD = 0.5·D_KL(P || M) + 0.5·D_KL(Q || M), M = (P+Q)/2.

Frequently Asked Questions

How does the Kullback-Leibler Divergence D_KL(P || Q), Cross-Entropy & Jensen-Shannon Calculator compute the solution?

It evaluates D_KL(P || Q) = Σ P(i) × log₂(P(i) / Q(i)); JSD = 0.5·D_KL(P || M) + 0.5·D_KL(Q || M), M = (P+Q)/2 directly in your browser and updates three custom SVG visualizations in real time.

Can I solve backwards for an input parameter using Goal Seek?

Yes. Click "🎯 Goal Seek" in the results toolbar to solve backwards for any numeric input field.

Does this calculator run locally offline?

Yes. All calculations run 100% locally in your browser and work offline via the CalculatorGraph PWA.

Mathematics & Statistics · Machine Learning & Classification Metrics

Kullback-Leibler Divergence D_KL(P || Q), Cross-Entropy & Jensen-Shannon Calculator: inputs, method, and how to read the result

Compute H(P), Cross-Entropy H(P,Q), D_KL(P||Q), and symmetric Jensen-Shannon Divergence.

Choose inputs for the case you are checking

The defaults are illustrative. Replace them with values for your case and keep the units shown beside each field.

  • True Distribution P1: True probability P(1) of outcome 1 (P(0) = 1 − P(1)).
  • Model Distribution Q1: Approximating probability Q(1) of outcome 1 (Q(0) = 1 − Q(1)).

What the calculation includes

With the default values shown, Forward KL Divergence D_KL(P || Q) is 0.184 bits. The calculation follows KL & Jensen-Shannon Divergence Formula; the formula and variable definitions above show how the inputs combine.

One-input check: With the other defaults held constant, changing True Distribution P1 from 0.7 P(1) to 0.77 P(1) changes Forward KL Divergence D_KL(P || Q) from 0.184 bits to 0.307 bits. This is a comparison inside the calculator's stated model, not a prediction.

Where to use caution

  • Uses exact analytical or standard numerical analysis formulations from NIST DLMF.
  • Double-precision floating-point arithmetic has ~15–17 significant decimal digits of precision.

Scope: Global Mathematical & Statistical Standard Check the assumptions and limitations against your situation before relying on the estimate.

More in Mathematics & Statistics: Scientific Calculator, Statistics & Standard Deviation Calculator, Permutations, Combinations, Probability, Sample Size & P-Value Calculator.