Isaac Scientific Publishing

Psychology Research and Applications

Using Statistics from Binary Variables to Detect Data Anomalies, Even Possibly Fraudulent Research

Download PDF (428 KB) PP. 112 - 118 Pub. Date: December 6, 2019

DOI: 10.22606/pra.2019.14004

Author(s)

  • Walter R. Schumm*
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, Manhattan, KS 66506, USA
  • Duane W. Crawford
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, Manhattan, KS 66506, USA
  • Lorenza Lockett
    Department of Sociology, Anthropology, and Social Work, College of Arts and Sciences, Kansas State University, Manhattan, KS 66506, USA

Abstract

Heathers and his colleagues have proposed a variety of tests to detect inconsistencies in research data, including the GRIM, SPRITE, DEBIT, and RIVETS tests. Here we focus on relatively simple ways of examining binary data for results that are impossible or results that feature inconsistencies, using binomial tests to evaluate whether anomalous results could be explained as random typographical errors. Hypothetical data are used to illustrate our suggested procedures. Advantages and limitations of the approaches are discussed.

Keywords

Research methods; detecting fraudulent research; binary tests; binomial tests; RIVETS, DEBIT, SPRITE, GRIM, and GRIMMER tests

References

[1] Anaya, J. (2016). The GRIMMER test: A method for testing the validity of repeated measures of variability. PeerJ Preprints, 4, e2400v1.

[2] Brown, N. J., & Heathers, J. A. (2017). The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science, 8(4), 363-369.

[3] Brown, N. J., & Heathers, J. A. (2019). Rounded Input Variables, Exact Test Statistics (RIVETS): A technique for detecting hand-calculated results in published research. Unpublished paper. Bouve College of Health Sciences, Northeastern University, 360 Huntington Avenue, Boston, MA, USA 02115.

[4] Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. Plos One, 4(5), e5738.

[5] Heathers, J., Anaya, J., van der Zee, T., & Brown, N. J. L. (2018). Recovering data from summary statistics: Sample parameter reconstruction via iterative techniques (SPRITE). Peerj Preprints. doi.org/10.7287/ peerj.preprints.26968v1.

[6] Heathers, J. A., & Brown, N. J. (2019). DEBIT: A simple consistency test for binary data. Unpublished paper. Bouve College of Health Sciences, Northeastern University, 360 Huntington Avenue, Boston, MA. USA 02115.

[7] Regnerus, M. (2012). How different are the adult children of parents who have same-sex relationships? Findings from the New Family Structures Study. Social Science Research, 41, 752-770.

[8] Schumm, W. R. (2015). Navigating treacherous waters – One researcher’s 40 years of experience with controversial scientific research. Comprehensive Psychology, 4, 24, 1-40.

[9] Schumm, W. R. (2018). Making sense of probability: What formulas to use under different conditions. Biomedical Journal of Scientific and Technical Research, 3(1), 2983-2984.

[10] Schumm, W. R., Crawford, D. W., & Lockett, L. (2019). Patterns of means and standard deviations with binary variables: A key to detecting fraudulent research. Biomedical Journal of Scientific and Technical Research, in press.

[11] Schumm, W. R., & Crawford, D. W. (2019). Scientific consensus on whether LGBTQ parents are more likely (or not) to have LGBTQ children: An analysis of 72 social science reviews of the literature published between 2001 and 2017. Journal of International Women’s Studies, 20(7), 1-12.

[12] Schumm, W. R., Crawford, D. W., Higgins, M., Lockett, L., AlRashed, A., & Ateeq, A. B. (2018). Estimating the standard deviation from the range: A replication of analysis of demographic data reported in Marriage & Family Review, 2016-2017. Marriage & Family Review, 54, 777-792.

[13] Simonsohn, U. (2013). Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24(10), 1875-1888.

[14] Steen, R. G., Casadevall, A., & Fang, F. C. (2013). Why has the number of scientific retractions increased? PloS One, 8(7), e68397.

[15] Stern, A. M., Casadevall, A., Steen, R. G., & Fang, F. C. (2014). Financial costs and personal consequences of research misconduct resulting in retracted publications. Elife, 3, e02956.

[16] Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science, 7(6), 670-688.

[17] Warren, J. R. (2019). How much do you have to publish to get a job in a top sociology department? Or to get tenure? Trends over a generation. Sociological Science, 6, 172-196.