Which reliability coefficient is commonly used for nominal or ordinal category agreement?

Which reliability coefficient is commonly used for nominal or ordinal category agreement?

• A. Pearson's r
• B. Intraclass Correlation Coefficient (ICC)
• C. Cohen's Kappa
• D. Standard error of measurement (SEM)

Answer: (C)

Cohen's Kappa measures how much two raters agree on categorical classifications beyond what would be expected by chance. This makes it ideal for nominal data, where categories are distinct labels, and it can be adapted to ordinal data with a weighted version that accounts for the degree of disagreement (closer categories count as less discordance). Pearson's r is about the linear relationship between continuous variables, not agreement on category labels. The intraclass correlation coefficient is typically used for continuous measurements across raters and often involves more than two raters or repeated measurements. The standard error of measurement focuses on the precision of a single observed score, not the level of agreement between different raters. So for assessing how well two raters categorize items into categories, Cohen's Kappa is the most appropriate choice.

Cohen's Kappa measures how much two raters agree on categorical classifications beyond what would be expected by chance. This makes it ideal for nominal data, where categories are distinct labels, and it can be adapted to ordinal data with a weighted version that accounts for the degree of disagreement (closer categories count as less discordance).

Pearson's r is about the linear relationship between continuous variables, not agreement on category labels. The intraclass correlation coefficient is typically used for continuous measurements across raters and often involves more than two raters or repeated measurements. The standard error of measurement focuses on the precision of a single observed score, not the level of agreement between different raters.

So for assessing how well two raters categorize items into categories, Cohen's Kappa is the most appropriate choice.