Calculate inter-rater reliability statistics

This function will calculate inter-rater reliability statistics for each code. The resulting data.frame will have the following columns:

Usage

irr(
  qda_data,
  codings = qda_data$get_codings(),
  min_ratings = 5,
  coders,
  include_zero_codes = TRUE
)

qda_data: a qda object.
codings: data frame of the codings on which to calculate IRR. This allows for calculating IRR metrics on a subset of the codings (e.g. select only a few coders).
min_ratings: the minimal number of text documents that are double coded for the IRR statistics to be calculated.
coders: a character vector of coders to include in the IRR calculations. If omitted then all coders will be used.
include_zero_codes: include text documents where none of the coders applied a code to that text.

a data.frame with inter-rater reliability (IRR) statistics for each code (see qda_data$get_codes()) and pair of coders.

code - the code.
n_text - total number of text documents coded.
n_double_coded - the number of text documents with at least two coders.
n_codes - the total number of times this code has been used.
n_text_with_code - the number of text documents with this code.
pra - Exact percent rater agreement. This compares the number of times the code was used for each text document.
icc1, icc2, icc3, icc1k, icc2k, icc3k - Intraclass correlation coefficient. See psych::ICC for more details.