Cohen’s Kappa (κ) Two-Rater Inter-Annotator Agreement Calculator

Compute chance-corrected inter-rater agreement κ and Landis-Koch benchmark tier.

Cohen’s Kappa (κ) Inter-Rater Agreement
0.759 κ
Raw Observed Agreement (p_o %) 88.333 %
Expected Chance Agreement (p_e %) 51.667 %
Total Items Rated (N) 120 items
Inputs
⚡ Scenarios:
cell a
Count where Rater 1 = Yes and Rater 2 = Yes (a).
cell b
Count where Rater 1 = Yes and Rater 2 = No (b).
cell c
Count where Rater 1 = No and Rater 2 = Yes (c).
cell d
Count where Rater 1 = No and Rater 2 = No (d).
Results & Graph
Cohen’s Kappa (κ) Inter-Rater Agreement
0.759 κ
Raw Observed Agreement (p_o %)
88.333 %
Expected Chance Agreement (p_e %)
51.667 %
Total Items Rated (N)
120 items

Cohen’s Kappa (κ) Inter-Rater Agreement

Graph 1 • Cohen’s Kappa (κ) Agreement • Domain Architecture & Live Telemetry Maps Both Positive Cell A, Rater1 Yes Rater2 No Cell B, Rater1 No Rater2 Yes Cell C through Cohen’s Kappa (κ) Agreement Formula.
Cohen’s Kappa (κ) Inter-Rater : 0.759 κ Cohen’s Kappa (κ) Agreement • Invariants & Metrics Cohen’s Kappa (κ) Inter-Rate 0.759 κ Raw Observed Agreement (p_o 88.333 % Expected Chance Agreement (p 51.667 % Total Items Rated (N) 120 items
Graph 2 • Cohen’s Kappa (κ) Inter-Rater Agreement Across Both Positive Cell A Sweep (40% → 160%) Evaluates p_o = (a + d) / N; p_e = ((a+b)(a+c) + (c+d)(b+d)) / N²; κ = (p_o − across 7 operating points (current input highlighted).
0 κ 0.550 κ 1.10 κ 0.608 16.80 0.684 25.20 0.729 33.60 0.759 42 0.779 50.40 0.795 58.80 0.807 67.20
Graph 3 • Joint Sensitivity: Both Positive Cell A × Rater1 Yes Rater2 No Cell B Compares Cohen’s Kappa (κ) Inter-Rater Agreement when Rater1 Yes Rater2 No Cell B is shifted +25% (solid) vs. −25% (dashed).
0 κ 0.550 κ 1.10 κ --- Rater1 Yes Rater2 No Cell B @ 75% 0.683 -30% 0.714 -15% 0.736 +0% 0.754 +15% 0.768 +30%
Summary: Observed agreement is 88.33% vs. 51.67% chance agreement, yielding Cohen’s κ = 0.7586 (Substantial (0.61–0.80)).
Summary: Observed agreement is 88.33% vs. 51.67% chance agreement, yielding Cohen’s κ = 0.7586 (Substantial (0.61–0.80)).
⚡ Send result to: Scientific →Statistics & SD →Probability & P-Value →
Interactive chart & math engine powered by CalculatorGraph

Formula & Mathematical Methodology: Cohen’s Kappa (κ) Agreement Formula

po = (a + d) / N
pe = ((a+b)(a+c) + (c+d)(b+d)) / N²
κ = po − pe1 − pe
p_o = (a + d) / N; p_e = ((a+b)(a+c) + (c+d)(b+d)) / N²; κ = (p_o − p_e) / (1 − p_e)
  • bothPositiveCellA: Both Positive Cell A (cell a)
  • rater1YesRater2NoCellB: Rater1 Yes Rater2 No Cell B (cell b)
  • rater1NoRater2YesCellC: Rater1 No Rater2 Yes Cell C (cell c)
  • bothNegativeCellD: Both Negative Cell D (cell d)

Rounding: Internal floating-point calculations run in 64-bit IEEE-754 double precision; display values round to 2–4 decimal places.

Worked Example: Worked Mathematical Example: Cohen’s Kappa (κ) Agreement

Compute chance-corrected inter-rater agreement κ and Landis-Koch benchmark tier.

  1. Specify the function parameters, matrix entries, or confusion matrix counts.
  2. Apply the closed-form or iterative formula: p_o = (a + d) / N; p_e = ((a+b)(a+c) + (c+d)(b+d)) / N²; κ = (p_o − p_e) / (1 − p_e).
  3. Examine the primary result, error bounds, and 3 interactive SVG plots.
Result: Computes Cohen’s Kappa (κ) Agreement via p_o = (a + d) / N; p_e = ((a+b)(a+c) + (c+d)(b+d)) / N²; κ = (p_o − p_e) / (1 − p_e).

Frequently Asked Questions

How does the Cohen’s Kappa (κ) Two-Rater Inter-Annotator Agreement Calculator compute the solution?

It evaluates p_o = (a + d) / N; p_e = ((a+b)(a+c) + (c+d)(b+d)) / N²; κ = (p_o − p_e) / (1 − p_e) directly in your browser and updates three custom SVG visualizations in real time.

Can I solve backwards for an input parameter using Goal Seek?

Yes. Click "🎯 Goal Seek" in the results toolbar to solve backwards for any numeric input field.

Does this calculator run locally offline?

Yes. All calculations run 100% locally in your browser and work offline via the CalculatorGraph PWA.

Mathematics & Statistics · Machine Learning & Classification Metrics

Cohen’s Kappa (κ) Two-Rater Inter-Annotator Agreement Calculator: inputs, method, and how to read the result

Compute chance-corrected inter-rater agreement κ and Landis-Koch benchmark tier.

Choose inputs for the case you are checking

The defaults are illustrative. Replace them with values for your case and keep the units shown beside each field.

  • Both Positive Cell A: Count where Rater 1 = Yes and Rater 2 = Yes (a).
  • Rater1 Yes Rater2 No Cell B: Count where Rater 1 = Yes and Rater 2 = No (b).
  • Rater1 No Rater2 Yes Cell C: Count where Rater 1 = No and Rater 2 = Yes (c).
  • Both Negative Cell D: Count where Rater 1 = No and Rater 2 = No (d).

What the calculation includes

With the default values shown, Cohen’s Kappa (κ) Inter-Rater Agreement is 0.759 κ. The calculation follows Cohen’s Kappa (κ) Agreement Formula; the formula and variable definitions above show how the inputs combine.

One-input check: With the other defaults held constant, changing Both Positive Cell A from 42 cell a to 46.2 cell a changes Cohen’s Kappa (κ) Inter-Rater Agreement from 0.759 κ to 0.77 κ. This is a comparison inside the calculator's stated model, not a prediction.

Where to use caution

  • Uses exact analytical or standard numerical analysis formulations from NIST DLMF.
  • Double-precision floating-point arithmetic has ~15–17 significant decimal digits of precision.

Scope: Global Mathematical & Statistical Standard Check the assumptions and limitations against your situation before relying on the estimate.

More in Mathematics & Statistics: Scientific Calculator, Statistics & Standard Deviation Calculator, Permutations, Combinations, Probability, Sample Size & P-Value Calculator.