Kappa for agreement with multiple raters

Cohen's kappa and weighed kappa for multiple raters, adapted from the `psych::cohen.kappa` function (Revelle, 2020). Light's method is used to combine the scores from multiple raters.

kappa(
  data,
  weight = NULL,
  confint = FALSE,
  alpha = 0.05,
  k = NULL,
  n = NULL,
  ...
)

Arguments

data

A `data.frame` or table with equal number of columns and rows. Or `data.frame` that contains the scores for each rater in each column.

weight

matrix of weights for the weighed kappa.

confint

Logical indicator for confidence interval

alpha

Confidence interval level, default = 0.05.

k

number of raters; default `k = ncol(data)`.

n

sample size; default `n = nrow(data)`.

...

options for sumtable if `is.data.frame(data)`

Value

vector

References

Revelle, W. (2020) psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA, https://CRAN.R-project.org/package=psych Version = 2.0.12,. Light, R. J. (1971) Measures of response agreement for qualitative data: Some generalizations and alternatives, Psychological Bulletin, 76, 365-377.

Examples

df <- data.frame(r1=factor(c(1,0,1,0,0,1,1,0,0,0,1,1,0,1,1)),
                 r2=factor(c(1,1,1,1,0,1,1,0,0,0,1,1,0,1,0)),
                 r3=factor(c(1,1,1,0,0,0,1,1,1,0,0,1,0,1,1)),
                 r4=factor(c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"), levels=c("0","1"))
kappa(df)
#>          kappa weighted kappa 
#>      0.2028986      0.2028986 
kappa(table)
#>          kappa weighted kappa 
#>      0.2028986      0.2028986 

df <- data.frame(r1=factor(c(1,2,2,0,3,3,1,0,3,0,2,2,0,3,1)),
                 r2=factor(c(1,1,1,0,3,3,1,0,1,0,2,2,0,2,1)),
                 r3=factor(c(1,1,1,3,3,2,1,0,1,0,2,2,0,3,1)),
                 r4=factor(c(1,2,1,0,3,3,1,0,3,0,2,2,0,2,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"), levels=c("0","1", "2", "3"))
kappa(df)
#>          kappa weighted kappa 
#>      0.6858639      0.7213395 
kappa(table)
#>          kappa weighted kappa 
#>      0.6858639      0.7213395 
kappa(df, confint = TRUE)
#>                    kappa     lower     upper
#> kappa          0.6858639 0.5691753 0.8025525
#> weighted kappa 0.7213395 0.5689155 0.8737636
kappa(table)
#>          kappa weighted kappa 
#>      0.6858639      0.7213395