Evaluation of Cognitive Processing in Redundant Audio-Visual Signals

Quebec City, Canada

Fox, E.L., Glavan, J.J. and Houpt, J.W. (2014). Evaluation of cognitive processing in redundant audio-visual signals. In P. Bello, M. Guarini, M. McShane and B. Scassellati (Eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society (pp. 475-480). Quebec City, Canada: Cognitive Science Society.

ABSTRACT: Many natural stimuli that we encounter have both audio and visual components.  Furthermore, when an audio and a visual signal co-occur, then they are often due to the same cause.  Given these facts, it is possible that perceptual processes would treat co-occurring audio and visual stimuli as evidence for a single decision.  Perceptual processes that pool separate sources of information toward making a single detection are known as “coactive.”  In contrast, perceptual processes may treat the audio and visual processes individually.  Naively, one might assume that if people are faster to detect audio-visual stimuli than either single modality stimulus, then they must be using a coactive process.  However, Raab (1962) demonstrated that this pattern of results could follow from independent parallel perception of each modality. Miller (1982) proposed an inequality that, when violated, could be used to distinguish between independent parallel and coactive processing.  Using this inequality, he found group level evidence for coactive processing of audio/visual stimuli. In the subsequent years, more sophisticated, individual level analyses have been developed for distinguishing parallel and coactive processing.  This more modern analysis framework is known as Systems Factorial Technology (SFT; Townsend & Nowaza, 1995; Houpt & Townsend, 2012).  In our study, we replicate Miller’s (1982) study, and then extend it so that we can apply SFT. Unlike Miller (1982), we do not find evidence of coactive processing; however, we do for some participants. By applying SFT, we are able to make more specific conclusions about the perceptual processes, and were able to make conclusions about individuals, not just the group. 


Houpt, J. W. & Townsend, J.T. (2012).  Statistical measures for workload capacity analysis.  Journal of Mathematical Psychology, 56, 341-355.

Miller, J. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive psychology, 14(2), 247-279.

Raab, D. (1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24, 574-590.

Townsend, J.T., & Nozawa, G. (1995). Spatio-temporal properties of elementary perception: An investigation of parallel, serial and coactive theories. Journal of Mathematical Psychology, 39, 321-360.