Klaus Krippendorff 
Reliability of Generating Data [PDF ebook] 

All data are the result of human actions whether by experimentations, observations, or declarations. As such, the presumption of knowing what data are about is subject to imperfections that can affect the validity of research efforts. With calls for data-based research comes the need to assure the reliability of generated data. The reliability of converting texts into analyzable data has become a burning issue in several areas. However, this issue has been met by only a few limited, and sometimes misleading measures of the extent to which data can be trusted as surrogates of the phenomena of analytical interests. The statistic proposed by the author – "Krippendorff’s Alpha" – is widely used in the social sciences, not only where human judgements are involved but also where measurements are compared. The Reliability of Generating Data expands on the author’s seminal work in content analysis and develops methods for assessing the reliability of the kind of data that previously defied evaluations for this purpose. It opens with a discussion of the epistemology of reliable data, then presents the most basic alpha coefficient for the single-valued coding of predefined units. This largely familiar way of measuring reliability provides the platform for the succeeding chapters which start with an overview of alternative coefficients and then expand alpha one quality after another, including to cope with the reliabilities of multi-valued coding, segmenting texts into meaningful units, big data, and information retrievals. It also includes a chapter on how to diagnose and remedy imperfections and one on applicable standards, all converging on the statistical issues of the reliability of generating data.Features: Provides an overview of methods for assessing the reliability of generating data Expands a statistic proposed by the author, already widely used in the social sciences Includes many easy to follow numerical examples to illustrate the measures Written to be useful to beginning and advanced researchers from many disciplines, notably linguistics, sociology, psychometric and educational research, and medical science