Ecological Validity in NeuroIS Research: Theory, Evidence, and a Roadmap for Future Studies

Abstract

NeuroIS research involves the application of neuroscience knowledge and tools to investigate information systems (IS) phenomena and research questions. Ecological validity (EV) refers to the ability to generalize results from a laboratory study—usually controlled experiments—to real-world settings. Because NeuroIS research is typically conducted in laboratory settings and relies on relatively intrusive measurement tools (when compared to more traditional data collection instruments), the issue of EV in NeuroIS is critical. Its importance is underscored by the IS discipline’s commitment to both rigor and relevance. In this paper, we discuss the nature and dimensions of EV and why it is important, outline the major threats to EV, and discuss its current status in NeuroIS research. Our status presentation is based on an analysis of all 42 empirical NeuroIS papers published in the Association for Information Systems “basket-of-eight” journals in the period 2007-2023 (i.e., the entire period of existence of the NeuroIS field). To provide guidance for future research, we also outline a roadmap for NeuroIS researchers, showing how to increase EV through research design decisions and study execution. Establishing EV is a task to make NeuroIS studies more relevant to real-world problems. Thus, the present paper is a direct response to recent articles calling for more NeuroIS studies with direct relevance to business and society without sacrificing scientific rigor.