The Electronic Journal of Business Research Methods provides perspectives on topics relevant to research in the field of business and management
For general enquiries email administrator@ejbrm.com
Click here to see other Scholarly Electronic Journals published by API
For a range of research text books on this and complimentary topics visit the Academic Bookshop
Information about The European Conference on Research Methodology for Business and Management Studies is available here

linkedin-120 

 

twitter2-125 

 

fb_logo-125 

 
Journal Issue
Volume 17 Issue 1 / Mar 2019  pp1‑16

Editor: Ann Brown

Download PDF (free)

Participant Carelessness: Is It a Substantial Problem With Survey Data?  pp1‑16

Shelly Marasi, Alison Wall, Kristen Brewer

Look inside Download PDF (free)

Abstract

For decades, participant carelessness has been considered a problem in collecting data using surveys. Although participant carelessness cannot be disputed to exist, the impact it has on data quality or the level of influence or bias it produces in results is questionable. The main purpose of this paper is to determine whether participant carelessness is a substantial problem that significantly influences or biases the results of statistical analyses. This is accomplished by analyzing established management relationships through a comparison of the full, careful, and careless samples to determine the impact participant carelessness has on data results regarding correlations, t‑tests, and simple linear regressions. Four detection approaches were used to identify careless participants individually, in pairs, and in three method combinations. The second purpose of this paper is to use the resampled individual reliability (RIR) approach to detect careless participants and compare it to the individual reliability approach to determine whether the two approaches are fundamentally similar. Data were collected using Mechanical Turk (N = 678). Based on the findings, participant carelessness does not appear to be a substantial problem or demonstrate levels of bias in the results in this study. There are two significant differences between the full and careful samples with the t‑tests and the regression comparisons of fit statistics demonstrate the careful samples to have a weak improvement over the full sample; however, none indicate bias. The findings also suggest that the individual reliability and the RIR approaches are not entirely fundamentally similar. 

 

Keywords: participant carelessness, insufficient effort responding, careless responding, random responding

 

Share |