Information processing and eye tracking in survey pretesting
DOI :
https://doi.org/10.12712/rpca.v18i4.63562Résumé
This study aimed investigate the effectiveness of eye tracking associated with cognitive interview in identifying survey pretesting problems. A quasi-experiment was carried out, using eye tracking and cognitive interviews to collect the data. The subjects responded to an online questionnaire, and a cognitive interview was later carried out with pre-programmed and probing questions. In the experimental treatment, the eye movements were tracked. The questionnaire used was the General Decision-Making Style Inventory by Scott and Bruce. Thus, it was possible to verify that using eye tracking associated with cognitive interviews is an effective complement in identifying survey pretesting problems.
Téléchargements
Références
Aizpurua, E. (2020). Pretesting methods in cross-cultural research. The essential role of language in survey research, 129.
Agresti, A., & Finlay, B. (2012). Métodos estatísticos para as ciências sociais. Porto Alegre: Penso.
Anthonissen, L. (2020). Cognition in construction grammar: Connecting individual and community grammars. Cognitive linguistics, 31(2), 309-337. https://doi.org/10.1515/cog-2019-0023
Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186-3191.
Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public opinion quarterly, 71(2), 287-311. https://doi.org/10.1093/poq/nfm006
Bender, C. S. (2019). O processamento da informação online e a decisão de compra em social commerce. Dissertação (Mestrado em Administração) – Universidade Federal de Santa Maria, Santa Maria.
Campoamor, N. B., Guerrini, C. J., Brooks, W. B., Bridges, J. F., & Crossnohere, N. L. (2024). Pretesting Discrete-Choice Experiments: A Guide for Researchers. The Patient-Patient-Centered Outcomes Research, 1-12. https://doi.org/10.1007/s40271-024-00672-z
Casey, G., & Moran, A. (1989). The computational metaphor and cognitive psychology. The Irish Journal of Psychology, 10(2), 143-161. https://doi.org/10.1080/03033910.1989.10557739
Conrad, F. G., & Blair, J. (2004). Data quality in cognitive interviews: the case of verbal reports. Methods for testing and evaluating survey questionnaires, 67-87. https://doi.org/10.1002/0471654728.ch4
Cozby, P. C. (2003). Métodos de pesquisa em ciências do comportamento. São Paulo: Atlas.
DeMaio, T. J., & Landreth, A. (2004). Do different cognitive interview techniques produce different results?. Methods for testing and evaluating survey questionnaires, 89-108. https://doi.org/10.1002/0471654728.ch5
Duchowski, T. A. (2017). Eye tracking: methodology theory and practice. Springer International Publishing AG.
Eysenck, M. W., & Keane, M. T. (2017). Manual de Psicologia Cognitiva-7. Artmed Editora.
Forsyth, B. H., & Lessler, J. T. (2004). Cognitive laboratory methods: A taxonomy. Measurement errors in surveys, 393-418. https://doi.org/10.1002/9781118150382.ch20
Galesic, M. (2018). Use of eye tracking for studying survey response processes 1. In Social and behavioral research and the internet, 349-370, Routledge.
Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology, John Wiley & Sons.
Hair, J., Babin, B., Money, A., & Samouel, P. (2005). Fundamentos de métodos de pesquisa em administração. Bookman Companhia Ed.
Hilton, C. E. (2017). The importance of pretesting questionnaires: a field research example of cognitive pretesting the Exercise referral Quality of Life Scale (ER-QLS). International Journal of Social Research Methodology, 20(1), 21-34. https://doi.org/10.1080/13645579.2015.1091640
Ikart, E. M. (2018). Questionnaire Pretesting Methods: A Comparison of Cognitive Interviewing and Respondent Debriefing Vis-à-vis the Study of the Adoption of Decision Support Systems by Knowledge Workers. International Journal of Business & Information, 13(2). DOI: 10.6702/ijbi.201806_13(2).0001
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: from eye fixations to comprehension. Psychological review, 87(4), 329. https://psycnet.apa.org/doi/10.1037/0033-295X.87.4.329
Kassner, M., Patera, W., & Bulling, A. (2014). Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication. https://doi.org/10.1145/2638728.2641695
Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied cognitive psychology, 5(3), 213-236. https://doi.org/10.1002/acp.2350050305
Lachman, R., Lachman, J. L., & Butterfield, E. C. (2015). Cognitive psychology and information processing: An introduction. Psychology Press. https://doi.org/10.4324/9781315798844
Lenzner, T., Hadler, P., & Neuert, C. E. (2024). Cognitive Pretesting. GESIS, Leibniz Institute for the Social Sciences (GESIS – Survey Guidelines), version 3.0, Mannheim, Germany. DOI: 10.15465/gesis-sg_en_049
Lenzner, T., Hadler, P., & Neuert, C. (2023). An experimental test of the effectiveness of cognitive interviewing in pretesting questionnaires. Quality & Quantity, 57(4), 3199-3217. https://doi.org/10.1007/s11135-022-01489-4
Lenzner, T., Kaczmirek, L., & Galesic, M. (2014). Left feels right: A usability study on the position of answer boxes in web surveys. Social Science Computer Review, 32(6), 743-764. https://doi.org/10.1177/0894439313517532
Liu, M. S., & Wang, M. L. (2020). A multi‐level model of commitment‐based human resource practices on team performance: the team cognition model. Asia Pacific Journal of Human Resources, 58(3), 378-398. https://doi.org/10.1111/1744-7941.12220
Löbler, M. L., dos Reis, E., Nishi, J. M., & Tagliapietra, R. D. (2019). Inventário de estilos de tomada de decisão: Validação de instrumento no contexto brasileiro. Revista de Administração Unimep, 17(1).
Luan, J., Yao, Z., Zhao, F., & Liu, H. (2016). Search product and experience product online reviews: An eye-tracking study on consumers' review search behavior. Computers in Human Behavior, 65, 420-430. https://doi.org/10.1016/j.chb.2016.08.037
Maldonado, C. E. (2017). Positive affirmation of non-algorithmic information processing. Cinta de moebio, (60), 279-285. http://dx.doi.org/10.4067/S0717-554X2017000300279
Miller, K., Chepp, V., Willson, S., & Padilla, J. L. (Eds.). (2014). Cognitive interviewing methodology. John Wiley & Sons.
Neuert, C. (2016). Eye tracking in questionnaire pretesting. Thesis, Universität Mannheim, Baden-Württemberg.
Olivos, F., & Liu, M. (2024). ChatGPTest: opportunities and cautionary tales of utilizing AI for questionnaire pretesting. Field Methods. https://doi-org.ez47.periodicos.capes.gov.br/10.1177/1525822X241280574
Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Methods for testing and evaluating survey questionnaires, 1-22. https://doi.org/10.1002/0471654728.ch1
Presser, S., & Blair, J. (1994). Survey pretesting: Do different methods produce different results?. Sociological methodology, 73-104. https://doi.org/10.2307/270979
Sampieri, R. H., Collado, C. F., & Lucio, P. B. (2013). Metodologia de pesquisa. São Paulo: McGraw-Hill.
Scott, S. G., & Bruce, R. A. (1995). Decision-making style: The development and assessment of a new measure. Educational and psychological measurement, 55(5), 818-831. https://doi.org/10.1177/0013164495055005017
Story, D. A., & Tait, A. R. (2019). Survey research. Anesthesiology, 130(2), 192-202. https://doi.org/10.1097/ALN.0000000000002436
Tagliapietra, R. D. (2018). Investigando a influência das recomendações de produtos online no processo de decisão de compra sob a perspectiva Neuro-IS. Dissertação (Mestrado em Administração) – Universidade Federal de Santa Maria, Santa Maria.
Thelk, A. D., & Hoole, E. R. (2006). What are you thinking? Postsecondary student think-alouds of scientific and quantitative reasoning items. The Journal of General Education, 55(1), 17-39. https://doi.org/10.2307/27798035
Tontini, J., Malheiros, M. B., dos Reis Lehnhart, E., & Rossato, V. P. (2023). Pré-teste cognitivo: um panorama da produção científica internacional. Gestión Joven, 24(3), 12-31.
Tourangeau, R. (1984). Cognitive sciences and survey methods. Cognitive aspects of survey methodology: Building a bridge between disciplines, 15, 73-100.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.
Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Sage publications.
Willis, G. (2018). Cognitive interviewing in survey design: State of the science and future directions. The Palgrave handbook of survey research, 103-107. https://doi.org/10.1007/978-3-319-54395-6_14
Yarbus, A. L. (1967). Eye movements and vision. New York: Plenum Press.
Yin, R. K. (2016). Pesquisa qualitativa do início ao fim. Porto Alegre: Penso.
Téléchargements
Publiée
Numéro
Rubrique
Licence
Os autores mantêm os direitos autorais e concedem à revista o direito de primeira publicação, com o trabalho simultaneamente licenciado sob a Creative Commons Attribution License, permitindo o compartilhamento do trabalho com reconhecimento da autoria e publicação inicial nesta revista.
A Revista usa o CC BY. Essa licença permite que outras pessoas distribuam, remixem, adaptem e desenvolvam seu trabalho, mesmo comercialmente, desde que sejam creditadas pela criação original. Essa é a licença mais flexível oferecida. Recomendado para máxima disseminação e uso de materiais licenciados.
Os autores têm autorização para assumir contratos adicionais separadamente, para distribuição não-exclusiva da versão do trabalho publicada nesta revista (ex.: publicar em repositório institucional ou como capítulo de livro), sempre com reconhecimento de autoria e publicação inicial nesta revista.
A Revista ulitiliza o DOI (Digital Object Identifier ) desde 2007 O DOI com a finalidade de autenticar cada artigo, portanto seu conteúdo digital.
