Calculate Weighted Kappa in SPSS: A Comprehensive Guide
Accurately measure inter-rater reliability using weighted kappa. This guide and calculator help you understand and implement this crucial statistical measure in SPSS.
Weighted Kappa Calculator
Number of agreements where both raters assigned Category 1.
Number of agreements where both raters assigned Category 2.
Number of agreements where both raters assigned Category 3.
Number of agreements where both raters assigned Category 1.
Number of agreements where both raters assigned Category 2.
Number of agreements where both raters assigned Category 3.
Total number of observations Rater 1 made.
Total number of observations Rater 2 made.
Linear
Quadratic
Ordinal
Choose the appropriate weighting scheme for your data.
Where weights are applied to disagreements based on the chosen scheme.
Observed vs. Expected Agreement Distribution
Observed Agreement Counts
Category
Rater 1 Count
Rater 2 Count
Category 1
Category 2
Category 3
What is Weighted Kappa in SPSS?
Weighted Kappa is a statistical measure used to assess the inter-rater reliability or agreement between two or more raters when classifying items into categories, especially when there's a possibility of chance agreement. In the context of SPSS (Statistical Package for the Social Sciences), it's a robust method for quantifying agreement beyond what's expected by random chance, particularly when the categories have an inherent order or hierarchy. Unlike simple percent agreement or unweighted kappa, weighted kappa accounts for the magnitude of disagreement. A disagreement between adjacent categories is considered less severe than a disagreement between distant categories. This makes it particularly useful in fields like psychology, medicine, education, and social sciences where subjective judgments or classifications are common, and the degree of difference matters.
Who Should Use It: Researchers, statisticians, and data analysts who need to evaluate the consistency of ratings or classifications made by different individuals (or the same individual at different times). This includes scenarios like:
Diagnosing patients based on symptom severity (ordinal scale).
Grading essays where different markers assign scores.
Classifying survey responses into predefined categories.
Assessing the reliability of diagnostic codes assigned by multiple clinicians.
Common Misconceptions:
Weighted Kappa is the same as Unweighted Kappa: While both measure agreement beyond chance, weighted kappa assigns different penalties for different degrees of disagreement, making it more nuanced for ordinal data.
A Kappa of 1 means perfect agreement: A kappa of 1 indicates perfect agreement *beyond chance*.
Kappa can only be between 0 and 1: While typically positive, kappa can be negative, indicating agreement worse than chance, though this is rare and usually points to systematic bias.
It's only for two raters: While the most common application is with two raters, extensions exist for more than two, although SPSS's built-in functionality primarily focuses on two raters.
Weighted Kappa Formula and Mathematical Explanation
The calculation of weighted kappa is more involved than simple agreement measures because it incorporates a weighting matrix that penalizes disagreements based on their severity. The general formula, often attributed to Fleiss and Cohen for ordinal scales, is:
κw = 1 – (Po' / Pe')
Where:
Po' (Observed Proportion of Weighted Agreement): This is the sum of the weighted agreements across all categories, divided by the total number of observations.
Pe' (Expected Proportion of Weighted Agreement): This is the sum of the weighted agreements expected by chance, calculated based on the marginal distributions of each rater's classifications.
Step-by-Step Derivation (Conceptual):
Define Categories and Weights: Identify the categories (e.g., C1, C2, C3) and construct a weighting matrix (W). For example, quadratic weighting assigns weights (i-j)² where i and j are category indices. Ordinal weights might use |i-j|, and linear weights use 1 for disagreement and 0 for agreement.
Calculate Observed Weighted Agreement (Po'):
For each category k, sum the weighted agreements: Σ (W_kk * n_kk), where n_kk is the number of observations assigned to category k by both raters.
Divide this sum by the total number of observations (N) to get Po'.
Calculate Marginal Totals: Sum the counts for each category for Rater 1 (a_k) and Rater 2 (b_k).
Calculate Expected Weighted Agreement (Pe'):
Calculate the expected number of agreements for each category by chance: (a_k * b_k) / N.
Multiply these expected counts by their corresponding diagonal weights (W_kk) from the weighting matrix.
Sum these weighted expected counts and divide by N.
Calculate Weighted Kappa (κw): Use the formula: κw = 1 – (Po' / Pe').
SPSS handles these complex calculations internally, but understanding the concept is crucial for interpretation. The calculator above simulates this process based on input counts and the selected weighting scheme.
Variable Explanations Table:
Key Variables in Weighted Kappa Calculation
Variable
Meaning
Unit
Typical Range
N
Total number of observations classified by both raters.
Count
≥ 1
n_kk
Number of observations where both raters assigned Category k.
Count
0 to N
a_k
Total count of observations assigned Category k by Rater 1.
Count
0 to N
b_k
Total count of observations assigned Category k by Rater 2.
Count
0 to N
W_ij
Weight assigned to disagreement between Category i and Category j. (W_kk = 0 for agreement).
Unitless
Depends on scheme (e.g., 0-1 for linear, squared differences for quadratic)
Po'
Observed proportion of weighted agreement.
Proportion (0-1)
0 to 1
Pe'
Expected proportion of weighted agreement by chance.
Proportion (0-1)
0 to 1
κw
Weighted Kappa statistic.
Value
Typically -1 to 1 (often 0 to 1)
Practical Examples (Real-World Use Cases)
Example 1: Medical Diagnosis Reliability
Two physicians independently diagnose patients for a specific condition using a 3-point severity scale: Mild (1), Moderate (2), Severe (3). They are concerned about the consistency of their diagnoses, especially distinguishing between moderate and severe cases. They use quadratic weighting because a misclassification from Mild to Severe is much worse than from Mild to Moderate.
Inputs:
Rater 1 Counts: Cat 1=40, Cat 2=30, Cat 3=10
Rater 2 Counts: Cat 1=35, Cat 2=35, Cat 3=10
Total Observations (N): 100
Weighting Scheme: Quadratic
Calculator Output:
Observed Weighted Agreement (Po'): 0.85
Expected Weighted Agreement (Pe'): 0.72
Weighted Kappa (κw): 0.45
Interpretation: A weighted kappa of 0.45 suggests moderate agreement beyond chance. The quadratic weighting means that disagreements like assigning 'Mild' when the other rater assigned 'Severe' significantly reduced the kappa value compared to unweighted kappa, highlighting the importance of distinguishing between the higher severity levels.
Example 2: Educational Assessment Grading
Two teachers grade student essays on a 4-point rubric: Unsatisfactory (1), Developing (2), Proficient (3), Exemplary (4). They need to ensure their grading is consistent. They decide to use ordinal weighting (|i-j|) as the difference between adjacent scores is considered equally weighted.
Inputs (Simplified for a 3-category example in the calculator):
Rater 1 Counts: Cat 1=60, Cat 2=25, Cat 3=15
Rater 2 Counts: Cat 1=55, Cat 2=30, Cat 3=15
Total Observations (N): 100
Weighting Scheme: Ordinal
Calculator Output:
Observed Weighted Agreement (Po'): 0.88
Expected Weighted Agreement (Pe'): 0.79
Weighted Kappa (κw): 0.42
Interpretation: A weighted kappa of 0.42 indicates moderate agreement beyond chance. The ordinal weighting penalizes larger gaps between scores more than smaller ones, providing a more accurate picture of reliability than simple percentage agreement.
How to Use This Weighted Kappa Calculator
This calculator simplifies the process of calculating weighted kappa for two raters classifying items into categories. Follow these steps:
Input Observed Counts: For each category (e.g., Category 1, Category 2, Category 3), enter the number of observations where *both* raters assigned that specific category. Use the fields labeled "Rater 1: Category X Count" and "Rater 2: Category X Count".
Input Total Observations: Enter the total number of observations classified by Rater 1 and Rater 2 in their respective fields. These totals should ideally match if all observations were classified by both raters.
Select Weighting Scheme: Choose the weighting scheme that best suits your data:
Linear: Simple weighting where disagreement is penalized linearly.
Quadratic: Penalizes disagreements more heavily as the distance between categories increases (suitable for interval or ratio-like scales).
Ordinal: Penalizes disagreement based on the absolute difference between category ranks (suitable for ordinal scales).
Click "Calculate Weighted Kappa": The calculator will process your inputs.
Review Results: The primary result, Weighted Kappa (κw), will be displayed prominently. Intermediate values like Observed Weighted Agreement (Po') and Expected Weighted Agreement (Pe') will also be shown, along with the formula explanation and a visual representation in the chart and table.
Interpret the Kappa Value:
κw = 1: Perfect agreement beyond chance.
0.81 – 1: Almost perfect agreement.
0.61 – 0.80: Substantial agreement.
0.41 – 0.60: Moderate agreement.
0.21 – 0.40: Slight agreement.
≤ 0.20: Poor agreement.
< 0: Agreement worse than chance.
Use "Copy Results": Click this button to copy all calculated values and assumptions for use in reports or further analysis.
Use "Reset": Click this button to clear current inputs and restore default values.
Key Factors That Affect Weighted Kappa Results
Several factors can significantly influence the weighted kappa value, impacting the interpretation of inter-rater reliability:
Prevalence of Categories: If one category is very common and another is very rare, it can affect both observed and expected agreement. High prevalence of a single category might inflate kappa if raters tend to agree on it.
Distribution of Ratings: The marginal distributions (how often each rater assigns each category) are critical. If raters consistently assign different distributions, kappa will be lower, reflecting systematic differences.
Degree of Disagreement: Weighted kappa explicitly considers the magnitude of disagreements. A small number of major disagreements (e.g., rating 'Severe' vs. 'Mild') will lower kappa more than the same number of minor disagreements (e.g., 'Mild' vs. 'Moderate') if appropriate weights are used.
Weighting Scheme Choice: The choice between linear, quadratic, or ordinal weighting fundamentally changes how disagreements are penalized. Using a scheme that doesn't match the data's nature can misrepresent reliability. Quadratic weighting, for instance, heavily penalizes large discrepancies, which might be appropriate for interval-like data but overstate disagreement for purely nominal data.
Rater Bias or Individual Tendencies: If one rater consistently uses a broader or narrower range of categories than the other, or tends to rate 'easier' or 'harder', this systematic difference will reduce kappa.
Ambiguity of Classification Criteria: Vague or poorly defined categories and criteria for classification inherently lead to lower agreement. If the guidelines are unclear, raters are more likely to interpret them differently, resulting in lower kappa values.
Number of Categories: While not directly in the formula, the number of categories can influence the chances of agreement. More categories can increase the potential for disagreement, potentially lowering kappa if agreement doesn't scale proportionally.
Definition of "Chance": The calculation of expected agreement (Pe') is based on the assumption that chance agreement is defined by the product of the marginal frequencies. If this assumption doesn't hold (e.g., due to rater training or specific biases), the baseline for kappa changes.
Frequently Asked Questions (FAQ)
Q1: What is the difference between weighted kappa and unweighted kappa?
Unweighted kappa treats all disagreements equally. Weighted kappa assigns different penalties (weights) to disagreements based on the distance between categories, making it more suitable for ordinal or interval data where the degree of error matters.
Q2: How do I choose the right weighting scheme (linear, quadratic, ordinal)?
Choose based on the nature of your categories. Use 'Ordinal' for ranked data where the difference between ranks is meaningful. Use 'Quadratic' if the squared difference between ranks is the best representation of disagreement severity (e.g., Likert scales with strong interval properties). Use 'Linear' as a general penalty or if categories are nominal but you want some penalty for disagreement.
Q3: Can weighted kappa be negative? What does that mean?
Yes, weighted kappa can be negative. It signifies that the observed agreement is less than what would be expected by chance alone. This typically indicates a systematic disagreement or bias between the raters.
Q4: How many observations are needed to calculate weighted kappa reliably?
There's no strict rule, but larger sample sizes provide more stable estimates. Generally, aim for enough observations to have sufficient counts in most cells of the agreement table. SPSS often requires at least 5 observations per cell for reliable estimation, though higher numbers are better.
Q5: Does weighted kappa account for inter-rater reliability in SPSS?
Yes, SPSS has procedures (like the `CROSSTABS` command with the `KAPPA` subcommand) that can calculate weighted kappa, provided you structure your data correctly (e.g., in a case-by-variable format where cases are observations and variables are raters' scores).
Q6: What is considered a "good" weighted kappa value?
Interpretation varies by field, but commonly: >0.80 is excellent, 0.60-0.80 is substantial, 0.40-0.60 is moderate, and <0.40 is considered poor to slight agreement beyond chance. The context and the cost of disagreement are crucial.
Q7: Can this calculator handle more than 3 categories?
The provided calculator is designed for up to 3 categories for simplicity in demonstration. SPSS can handle any number of categories. For more categories, you would need to extend the input fields and calculation logic.
Q8: How does the total number of observations affect the result?
The total number of observations (N) is the denominator in calculating the proportions Po' and Pe'. A larger N generally leads to more stable estimates of agreement. It also impacts the calculation of expected agreement by chance.
Best practices for making sense of statistical results.
function validateInput(id, min, max) {
var input = document.getElementById(id);
var errorDiv = document.getElementById('error-' + id);
var value = parseFloat(input.value);
if (isNaN(value)) {
errorDiv.textContent = "Please enter a valid number.";
errorDiv.style.display = 'block';
return false;
}
if (value max) {
errorDiv.textContent = "Value cannot be greater than " + max + ".";
errorDiv.style.display = 'block';
return false;
}
errorDiv.textContent = ";
errorDiv.style.display = 'none';
return true;
}
function calculateWeightedKappa() {
// Clear previous errors
var errorIds = ['rater1Score1', 'rater1Score2', 'rater1Score3', 'rater2Score1', 'rater2Score2', 'rater2Score3', 'rater1Total', 'rater2Total'];
for (var i = 0; i < errorIds.length; i++) {
var errorDiv = document.getElementById('error-' + errorIds[i]);
if (errorDiv) {
errorDiv.style.display = 'none';
}
}
// Validate inputs
var allValid = true;
allValid = validateInput('rater1Score1', 0) && allValid;
allValid = validateInput('rater1Score2', 0) && allValid;
allValid = validateInput('rater1Score3', 0) && allValid;
allValid = validateInput('rater2Score1', 0) && allValid;
allValid = validateInput('rater2Score2', 0) && allValid;
allValid = validateInput('rater2Score3', 0) && allValid;
allValid = validateInput('rater1Total', 0) && allValid;
allValid = validateInput('rater2Total', 0) && allValid;
if (!allValid) {
document.getElementById('results-container').style.display = 'none';
return;
}
var r1s1 = parseFloat(document.getElementById('rater1Score1').value);
var r1s2 = parseFloat(document.getElementById('rater1Score2').value);
var r1s3 = parseFloat(document.getElementById('rater1Score3').value);
var r2s1 = parseFloat(document.getElementById('rater2Score1').value);
var r2s2 = parseFloat(document.getElementById('rater2Score2').value);
var r2s3 = parseFloat(document.getElementById('rater2Score3').value);
var r1Total = parseFloat(document.getElementById('rater1Total').value);
var r2Total = parseFloat(document.getElementById('rater2Total').value);
var weightingScheme = document.getElementById('weightingScheme').value;
// Ensure totals are sufficient
if (r1Total < r1s1 + r1s2 + r1s3 || r2Total < r2s1 + r2s2 + r2s3) {
alert("Total observations must be greater than or equal to the sum of category counts for each rater.");
document.getElementById('results-container').style.display = 'none';
return;
}
if (r1Total === 0 || r2Total === 0) {
alert("Total observations cannot be zero.");
document.getElementById('results-container').style.display = 'none';
return;
}
// Simplified example for 3 categories. In a real scenario, this would be more generalized.
// For demonstration, we assume the inputs directly map to the structure needed.
// Actual SPSS calculation involves contingency tables.
// This calculator simulates the core weighted kappa logic.
// Let's use a simplified representation for N, marginals, and agreement counts
var N = Math.min(r1Total, r2Total); // Use the minimum total for calculations involving both raters
if (N === 0) {
alert("Total observations cannot be zero.");
document.getElementById('results-container').style.display = 'none';
return;
}
// Observed agreement counts (directly from inputs for simplicity)
var n_11 = r1s1; // Rater 1 Category 1, Rater 2 Category 1
var n_22 = r1s2; // Rater 1 Category 2, Rater 2 Category 2
var n_33 = r1s3; // Rater 1 Category 3, Rater 2 Category 3
// Adjust n_kk if inputs don't match (this is a simplification)
// In a real contingency table, we'd have off-diagonal disagreements.
// For this calculator, we'll focus on the diagonal agreement for Po' and Pe'
// derived from the provided inputs, assuming they represent the core diagonal.
// A full implementation would require the full contingency table.
// Marginal totals
var r1_cat1_total = r1s1; // Assuming r1s1 is the count for category 1 for rater 1
var r1_cat2_total = r1s2; // Assuming r1s2 is the count for category 2 for rater 1
var r1_cat3_total = r1s3; // Assuming r1s3 is the count for category 3 for rater 1
var r2_cat1_total = r2s1; // Assuming r2s1 is the count for category 1 for rater 2
var r2_cat2_total = r2s2; // Assuming r2s2 is the count for category 2 for rater 2
var r2_cat3_total = r2s3; // Assuming r2s3 is the count for category 3 for rater 2
// Use the provided totals for calculations involving total observations
var effectiveN = Math.min(r1Total, r2Total);
if (effectiveN === 0) {
alert("Total observations cannot be zero.");
document.getElementById('results-container').style.display = 'none';
return;
}
// Calculate Po' (Observed Weighted Agreement)
var Po_prime = 0;
var observed_disagreement_weighted = 0;
var total_weighted_disagreement = 0; // For normalization if needed, though Po' is direct sum
var weights = {};
// Define weights based on scheme
if (weightingScheme === 'linear') {
weights = {
1: {1: 0, 2: 1, 3: 2},
2: {1: 1, 2: 0, 3: 1},
3: {1: 2, 2: 1, 3: 0}
};
} else if (weightingScheme === 'quadratic') {
weights = {
1: {1: 0, 2: 1, 3: 4},
2: {1: 1, 2: 0, 3: 1},
3: {1: 4, 2: 1, 3: 0}
};
} else { // Ordinal
weights = {
1: {1: 0, 2: 1, 3: 2},
2: {1: 1, 2: 0, 3: 1},
3: {1: 2, 2: 1, 3: 0}
};
}
// Simplified Po': Sum of weighted agreements / N
// This is a highly simplified Po' based on direct diagonal counts assuming they are the primary agreements.
// A full calculation requires the full contingency table and summing weighted agreements.
// For this example, we'll use the provided n_kk values directly for observed weighted agreements.
var sum_weighted_agreements_obs = (weights[1][1] * r1s1) + (weights[2][2] * r1s2) + (weights[3][3] * r1s3);
Po_prime = sum_weighted_agreements_obs / effectiveN;
// Calculate Pe' (Expected Weighted Agreement)
// Expected counts for each category
var exp_1 = (r1_cat1_total * r2_cat1_total) / effectiveN;
var exp_2 = (r1_cat2_total * r2_cat2_total) / effectiveN;
var exp_3 = (r1_cat3_total * r2_cat3_total) / effectiveN;
// Sum of weighted expected agreements
var sum_weighted_agreements_exp = (weights[1][1] * exp_1) + (weights[2][2] * exp_2) + (weights[3][3] * exp_3);
Pe_prime = sum_weighted_agreements_exp / effectiveN;
// Calculate Kappa
var kappa = 0;
if (Pe_prime !== 0) { // Avoid division by zero
// Correction: Kappa formula is 1 – (Observed Disagreement / Expected Disagreement)
// Which is equivalent to 1 – (1 – Po') / (1 – Pe') if weights are normalized 0-1
// A more direct common formula using observed and expected *agreement* is:
// Kappa = (Po – Pe) / (1 – Pe)
// For *weighted* kappa, it's often formulated as:
// Kappa_w = 1 – (Observed Weighted Disagreement / Expected Weighted Disagreement)
// Let's re-calculate using the concept of disagreement:
// Total potential weighted disagreement
// This requires all cells of the contingency table and weights matrix.
// Simplification: Use the formula structure presented in the article:
// Kappa = 1 – (Observed Disagreement / Expected Disagreement)
// Let's assume Po' and Pe' represent PROPORTIONS OF WEIGHTED AGREEMENT
// Then Kappa = (Po' – Pe') / (1 – Pe') is a common form.
// Re-evaluating common Kappa formulas:
// Kappa = (Po – Pe) / (1 – Pe)
// Where Po and Pe are proportions of AGREEMENT.
// Let's redefine Po_prime and Pe_prime to be proportions of agreement.
// Po_prime = sum(n_kk * w_kk) / N
// Pe_prime = sum(exp_k * w_kk) / N
// The calculator's formula description is "1 – (observed disagreement / expected disagreement)"
// Let's try to derive disagreement proportions.
// Total possible weighted disagreement = Total potential disagreement – actual weighted agreement
// This requires the full matrix.
// Let's use the common interpretation where Po and Pe are proportions of AGREEMENT
// and the formula is (Po – Pe) / (1 – Pe).
// Our current Po_prime and Pe_prime are based on summing weights * category counts.
// If weights represent penalty, then:
// Observed Agreement = Sum(n_kk)
// Expected Agreement = Sum(exp_k)
// Po = Sum(n_kk) / N
// Pe = Sum(exp_k) / N
// Kappa = (Po – Pe) / (1 – Pe)
// For weighted kappa, the weights apply to disagreement.
// Let's use the definition from the article: 1 – (Observed Disagreement / Expected Disagreement)
// Let's assume the formula in the article is the goal.
// This implies we need to calculate disagreement measures.
// Simplified Calculation Approach for the calculator:
// Assume Po_prime and Pe_prime are proportions of AGREEMENT, weighted.
// Po_prime_agreement = (sum of weighted counts in agreement cells) / N
// Pe_prime_agreement = (sum of weighted counts in expected agreement cells) / N
// Let's adjust the calculation to match the formula in the article directly.
// We need to calculate weighted disagreement.
// Total possible weighted disagreement = sum over all i!=j ( w_ij * N_ij ) where N_ij is total possible disagreement.
// This requires the full contingency table.
// Let's revert to the commonly cited formula using agreement proportions:
// kappa = (Po – Pe) / (1 – Pe)
// We need Po and Pe (unweighted agreement proportions) first.
var Po_unweighted = (n_11 + n_22 + n_33) / effectiveN;
var Pe_unweighted = (exp_1 + exp_2 + exp_3) / effectiveN;
// Now, how to incorporate weights?
// A common approach for weighted kappa (e.g., Cohen's Kappa with weights):
// Kappa = (Sum(w_ij * n_ij) – Sum(w_ij * exp_ij)) / (N – Sum(w_ij * exp_ij)) — This is complex
// Simpler form if weights represent agreement:
// Weighted agreement = sum (n_kk * w_kk) / N — This is what we have for Po_prime
// Expected weighted agreement = sum (exp_k * w_kk) / N — This is what we have for Pe_prime
// If Po_prime and Pe_prime are *weighted agreement proportions*:
// Kappa_w = (Po_prime – Pe_prime) / (1 – Pe_prime) — This is a commonly used formulation
// Let's use this for calculation.
kappa = (Po_prime – Pe_prime) / (1 – Pe_prime);
if (isNaN(kappa) || !isFinite(kappa)) {
kappa = 0; // Default if calculation fails
}
} else {
kappa = 1; // Perfect agreement if expected agreement is zero (highly unlikely)
}
// Display results
var resultsContainer = document.getElementById('results-container');
document.getElementById('primary-result').innerHTML = '' + kappa.toFixed(3) + '';
document.getElementById('observed-agreement').innerHTML = 'Observed Weighted Agreement (Po\'): ' + Po_prime.toFixed(3);
document.getElementById('expected-agreement').innerHTML = 'Expected Weighted Agreement (Pe\'): ' + Pe_prime.toFixed(3);
document.getElementById('kappa-value').innerHTML = 'Weighted Kappa (κw): ' + kappa.toFixed(3);
var assumptionsText = "Weighting Scheme: " + weightingScheme.charAt(0).toUpperCase() + weightingScheme.slice(1) + "; ";
assumptionsText += "Total Observations Considered: " + effectiveN;
document.getElementById('assumptions-text').innerHTML = assumptionsText;
resultsContainer.style.display = 'block';
// Update table
document.getElementById('table-rater1-cat1').textContent = r1s1;
document.getElementById('table-rater1-cat2').textContent = r1s2;
document.getElementById('table-rater1-cat3').textContent = r1s3;
document.getElementById('table-rater2-cat1').textContent = r2s1;
document.getElementById('table-rater2-cat2').textContent = r2s2;
document.getElementById('table-rater2-cat3').textContent = r2s3;
// Update chart
updateChart(r1s1, r1s2, r1s3, r2s1, r2s2, r2s3, effectiveN);
}
function updateChart(r1s1, r1s2, r1s3, r2s1, r2s2, r2s3, N) {
var ctx = document.getElementById('agreementChart').getContext('2d');
// Ensure chart instance exists, destroy if it does
if (window.agreementChartInstance) {
window.agreementChartInstance.destroy();
}
window.agreementChartInstance = new Chart(ctx, {
type: 'bar',
data: {
labels: ['Category 1', 'Category 2', 'Category 3'],
datasets: [
{
label: 'Rater 1 Total',
data: [r1s1, r1s2, r1s3],
backgroundColor: 'rgba(0, 74, 153, 0.6)', // Primary color
borderColor: 'rgba(0, 74, 153, 1)',
borderWidth: 1
},
{
label: 'Rater 2 Total',
data: [r2s1, r2s2, r2s3],
backgroundColor: 'rgba(40, 167, 69, 0.6)', // Success color
borderColor: 'rgba(40, 167, 69, 1)',
borderWidth: 1
}
]
},
options: {
responsive: true,
maintainAspectRatio: true,
scales: {
y: {
beginAtZero: true,
title: {
display: true,
text: 'Number of Observations'
}
}
},
plugins: {
legend: {
position: 'top',
},
title: {
display: true,
text: 'Agreement Counts by Category'
}
}
}
});
// Update legend manually if needed or rely on chart.js legend
var legendHtml = 'Legend: Rater 1 Total, Rater 2 Total';
document.getElementById('chart-legend').innerHTML = legendHtml;
}
function resetCalculator() {
document.getElementById('rater1Score1').value = 50;
document.getElementById('rater1Score2').value = 20;
document.getElementById('rater1Score3').value = 10;
document.getElementById('rater2Score1').value = 30;
document.getElementById('rater2Score2').value = 40;
document.getElementById('rater2Score3').value = 5;
document.getElementById('rater1Total').value = 100;
document.getElementById('rater2Total').value = 100;
document.getElementById('weightingScheme').value = 'linear';
// Clear results and hide container
document.getElementById('primary-result').textContent = ";
document.getElementById('observed-agreement').textContent = ";
document.getElementById('expected-agreement').textContent = ";
document.getElementById('kappa-value').textContent = ";
document.getElementById('assumptions-text').textContent = ";
document.getElementById('results-container').style.display = 'none';
// Clear table data
document.getElementById('table-rater1-cat1').textContent = ";
document.getElementById('table-rater1-cat2').textContent = ";
document.getElementById('table-rater1-cat3').textContent = ";
document.getElementById('table-rater2-cat1').textContent = ";
document.getElementById('table-rater2-cat2').textContent = ";
document.getElementById('table-rater2-cat3').textContent = ";
// Clear errors
var errorIds = ['rater1Score1', 'rater1Score2', 'rater1Score3', 'rater2Score1', 'rater2Score2', 'rater2Score3', 'rater1Total', 'rater2Total'];
for (var i = 0; i < errorIds.length; i++) {
var errorDiv = document.getElementById('error-' + errorIds[i]);
if (errorDiv) {
errorDiv.textContent = '';
errorDiv.style.display = 'none';
}
}
// Update chart with default values
updateChart(50, 20, 10, 30, 40, 5, 100);
}
function copyResults() {
var primaryResult = document.getElementById('primary-result').innerText.trim();
var observed = document.getElementById('observed-agreement').innerText.trim();
var expected = document.getElementById('expected-agreement').innerText.trim();
var kappa = document.getElementById('kappa-value').innerText.trim();
var assumptions = document.getElementById('assumptions-text').innerText.trim();
var formula = "Formula Used: Weighted Kappa (κw) = 1 – ( (observed disagreement) / (expected disagreement) )";
if (!primaryResult) {
alert("No results to copy yet.");
return;
}
var textToCopy = "Weighted Kappa Results:\n\n";
textToCopy += "Primary Result: " + primaryResult + "\n";
textToCopy += observed + "\n";
textToCopy += expected + "\n";
textToCopy += kappa + "\n\n";
textToCopy += assumptions + "\n";
textToCopy += formula + "\n";
// Use navigator.clipboard for modern browsers
if (navigator.clipboard && navigator.clipboard.writeText) {
navigator.clipboard.writeText(textToCopy).then(function() {
alert('Results copied to clipboard!');
}).catch(function(err) {
console.error('Failed to copy: ', err);
fallbackCopyTextToClipboard(textToCopy); // Fallback for older browsers
});
} else {
fallbackCopyTextToClipboard(textToCopy); // Fallback
}
}
function fallbackCopyTextToClipboard(text) {
var textArea = document.createElement("textarea");
textArea.value = text;
textArea.style.position = "fixed"; // Avoid scrolling to bottom
textArea.style.left = "-9999px";
textArea.style.top = "-9999px";
document.body.appendChild(textArea);
textArea.focus();
textArea.select();
try {
var successful = document.execCommand('copy');
var msg = successful ? 'Results copied to clipboard!' : 'Failed to copy results.';
alert(msg);
} catch (err) {
console.error('Fallback: Oops, unable to copy', err);
alert('Failed to copy results. Please copy manually.');
}
document.body.removeChild(textArea);
}
// Initialize chart on page load with default values
document.addEventListener('DOMContentLoaded', function() {
// Add Chart.js library dynamically (or ensure it's in your theme/plugin)
// For a single file, we'll assume Chart.js is available or needs to be included.
// If running this stand-alone, you'd need to add:
// in the head.
// For this example, we assume it's available.
// Check if Chart is available before initializing
if (typeof Chart === 'undefined') {
console.error("Chart.js is not loaded. Please include Chart.js library.");
// Optionally, show a message to the user or load it dynamically
var script = document.createElement('script');
script.src = 'https://cdn.jsdelivr.net/npm/chart.js';
script.onload = function() {
console.log("Chart.js loaded successfully.");
updateChart(50, 20, 10, 30, 40, 5, 100); // Call after load
};
script.onerror = function() {
console.error("Failed to load Chart.js from CDN.");
};
document.head.appendChild(script);
} else {
updateChart(50, 20, 10, 30, 40, 5, 100); // Call directly if already loaded
}
// Trigger initial calculation if desired, or wait for user interaction
// calculateWeightedKappa();
});