The default function returns the proportion of overall agreement between two or more raters. Specific agreements can be obtained as well. Specific agreement (averages over discordant cells to correct for random rater combinations). When there are 2 categories, this is equal to the positive/negative agreement. When there are more than two categories, one can either look at the agreement for one category versus the others, for example very satisfied versus rest or for one category versus one specific other category, for example very satisfies versus not satisfied.

agreement(
data,
specific = NULL,
confint = FALSE,
alpha = 0.05,
n = nrow(data),
k = ncol(data),
b = 1000,
...
)

## Arguments

data

A data frame or table with equal number of columns and rows. Or a data frame that contains the scores for each rater in each column.

specific

A character vector indicating the category for which specific agreement should be calculated. If length(specific) == 1, the named factor level is compared to all others, if length(specific) == 2 the specific agreement is of the first category level is compared to the second category level. If specific = "positive" or specific = "negative" and scores are dichotomous, the positive or negative agreement will be returned.

confint

logical vector if confidence interval for agreements are computed.

alpha

the confidence interval level

n

sample size; default n = nrow(data)

k

number of raters; default k = ncol(data)

b

number of bootstrap iterations for bootstrapped CI

...

options for sumtable if data = data.frame

## Value

An S3 object containing the proportion of overall agreement.

## Details

De confidence intervals are obtained and adjusted with a Fleis continuity correction. For the specific agreements with polytomous data, the confidence intervals are bootstrapped.

## Examples

#dichotomous
df <- data.frame(r1=factor(c(1,0,1,0,0,1,1,0,0,0,1,1,0,1,1)),
r2=factor(c(1,1,1,1,0,1,1,0,0,0,1,1,0,1,0)),
r3=factor(c(1,1,1,0,0,0,1,1,1,0,0,1,0,1,1)),
r4=factor(c(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"),
levels=c("0","1"))
agreement(table)
#> overall agreement
#>         0.6333333
agreement(table, specific = "1")
#>     overall agreement specific agreement: 1
#>             0.6333333             0.7317073
agreement(table, specific = "1", confint = TRUE)
#>                               p     lower     upper
#> overall agreement     0.6333333 0.2154740 0.8617275
#> specific agreement: 1 0.7317073 0.6199105 0.8259595
agreement(table, specific = "positive")
#>  overall agreement positive agreement
#>          0.6333333          0.4210526
agreement(df)
#> overall agreement
#>         0.6333333
agreement(df, specific = "1")
#>     overall agreement specific agreement: 1
#>             0.6333333             0.7317073
agreement(df, specific = "positive")
#>  overall agreement positive agreement
#>          0.6333333          0.4210526
agreement(df, specific = "negative")
#>  overall agreement negative agreement
#>          0.6333333          0.7317073
#polytomous
df <- data.frame(r1=factor(c(1,2,2,0,3,3,1,0,3,0,2,2,0,3,1)),
r2=factor(c(1,1,1,0,3,3,1,0,1,0,2,2,0,2,1)),
r3=factor(c(1,1,1,3,3,2,1,0,1,0,2,2,0,3,1)),
r4=factor(c(1,2,1,0,3,3,1,0,3,0,2,2,0,2,1)))
table <- sumtable(df=df, ratings=c("r1", "r2", "r3", "r4"),
levels=c("0","1", "2", "3"))
agreement(table, specific = c("3", "1"))
#>          overall agreement specific agreement: 3 vs 1
#>                  0.7666667                  0.8461538
agreement(table, specific = c("3", "1", "2"))
#> Warning: The specific agreement cannot be obtained for more than two categories; only the first argument is used and compared to all other category levels available. To avoid this warning use: specific = 3
#>     overall agreement specific agreement: 3
#>             0.7666667             0.6111111
agreement(table, specific = c(3))
#>     overall agreement specific agreement: 3
#>             0.7666667             0.6666667
agreement(table, specific = c("3", "1"), confint = TRUE)
#> Warning: input should be a data.frame to obtain confidence intervals for specific agreement with more than 2 factor levels; confinterval are not returned
#>          overall agreement specific agreement: 3 vs 1
#>                  0.7666667                  0.8461538
agreement(df, specific = c("3", "1"), confint = TRUE)
#>                                    p     lower     upper
#> overall agreement          0.7666667 0.6232767 0.8747149
#> specific agreement: 3 vs 1 0.8461538 0.4285714 1.0000000