Auditory Short-Term Memory Behaves Like Visual Short-Term Memory

Abstract

Are the information processing steps that support short-term sensory memory common to all the senses? Systematic, psychophysical comparison requires identical experimental paradigms and comparable stimuli, which can be challenging to obtain across modalities. Participants performed a recognition memory task with auditory and visual stimuli that were comparable in complexity and in their neural representations at early stages of cortical processing. The visual stimuli were static and moving Gaussian-windowed, oriented, sinusoidal gratings (Gabor patches); the auditory stimuli were broadband sounds whose frequency content varied sinusoidally over time (moving ripples). Parallel effects on recognition memory were seen for number of items to be remembered, retention interval, and serial position. Further, regardless of modality, predicting an item's recognizability requires taking account of (1) the probe's similarity to the remembered list items (summed similarity), and (2) the similarity between the items in memory (inter-item homogeneity). A model incorporating both these factors gives a good fit to recognition memory data for auditory as well as visual stimuli. In addition, we present the first demonstration of the orthogonality of summed similarity and inter-item homogeneity effects. These data imply that auditory and visual representations undergo very similar transformations while they are encoded and retrieved from memory. Memories are not exact representations of the past. But can we say that all our senses are equally reliable (or unreliable) sources for memory? We performed a series of experiments to test that proposition. Sound and light are processed by different receptors and neural pathways in the brain. Previous comparisons of auditory and visual memory have done little to place on equal footing the stimuli that will be remembered, limiting the ability to truly compare the two processes. However, using current knowledge of how these sensations are represented in the nervous system, we created auditory and visual stimuli of similar complexity and that undergo similar initial processing by the nervous system. We then used these well-matched stimuli to examine memory for studied lists of either auditory or visual items. Using behavioral measures and a computational model for list memory, we show that memory representations are altered similarly for both hearing and vision. We found that auditory and visual memory exhibit striking parallels in terms of how memory is affected by all the parameters we changed in this experiment. These results imply that auditory and visual short-term memory employ similar mechanisms.