Posters

A task-irrelevant, high-contrast stimulus can be used as a visual impulse signal to implement a functional non-invasive perturbation method to reveal working memory (WM) content. Stokes (2015) suggested that the impulse acts like a sonar signal used in echolocation, from which structural information (e.g., the surface of the ocean floor) can be derived. Similarly, in the brain the visual impulse reveals memoranda embedded in functional connectivity, which might by themselves be activity-silent. However, Barbosa et al. (2021) suggested that the impulse might only decrease non-WM-related EEG noise, thereby improving the ability to decode already-active memoranda. In this study, we sought to arbitrate between these two possibilities. We matched a task-irrelevant feature (spatial frequency) of a visual impulse with memory items (orientation gratings), while equalizing intensity and contrast. Better decoding of WM content in the match condition than in the no-match condition would suggest that the impulse interacts with the actual content within WM network, in line with activity-silent accounts. Conversely, if no differences between conditions are observed, this would fit with a noise reduction account, and suggest that WM might rely primarily on active storage. Results showed an advantage for matching impulses, supporting the former hypothesis that visual impulses work as a neural sonar. Further, although the visual impulse decreased average EEG variance, there was no difference between match and no-match conditions. We conclude that visual impulse perturbation reveals memoranda embedded in functional connectivity, in line with the idea that WM might rely on activity-silent states.

Recent studies suggest a tighter relationship between visual working memory (VWM), attention, and action than traditionally assumed. In two experiments using eye tracking, we tested the prediction that attention is biased more by VWM representations that are the direct target of an action plan, than by those that are not. In Experiment 1, on each trial, participants (N=52) memorized a geometric shape. Their memory of the shape was subsequently tested in a probe display. In the action condition, participants had to grip the probe when it matched the memorized shape, while, in the control condition, they had to grip another visual stimulus. Crucially, during the memory delay, a visual selection task was presented. Participant had to rapidly saccade towards a target letter. The same shape as the one held in memory was always present in the visual selection task display, and either contained the target (congruent trials) or contained a distractor (incongruent trials). Analysis of saccade latencies revealed greater attentional biases towards memory-matches in the action condition, as reflected by a significant interaction between congruency and condition. This suggests that intending to act on an object in VWM strengthens its representation. In Experiment 2, we investigated to what extent the effect relies on how well-defined the action plan is. In one condition, participants (N=51) knew precisely in advance which of three actions they had to perform on the memory probe, while in the other condition they knew that an action had to be performed, but not which one until the memory test. We again observed an interaction between congruency and condition, reflecting stronger interference from VWM representations for which the action was defined in advance. We conclude that action intention strengthens VWM representations.

Visual working memory depends on both material-specific posterior brain areas that support the representation of the to-be-maintained stimulus features and executive control areas in the prefrontal cortex (PFC). These two areas putatively support bottom-up (storage) vs. top-down (control) functions that rely on oscillations in the gamma and beta frequency bands, respectively. A previous intracranial EEG (iEEG) study combined representational similarity analysis (RSA) and deep neural networks (DNNs) as a model of the processing hierarchy along the ventral visual stream (VVS) and showed that maintenance of natural images relied on both high-level visual and semantic information. How selective attention prioritizes the representational formats of specific working memory contents and suppresses irrelevant contents remains an open question, however. Here, we addressed this issue using a multi-item working memory task involving a retro-cue. We recorded iEEG activity from patients with electrodes in the VVS, PFC, and/or hippocampus. We report a dissociation of the representational signatures across working memory stages: During encoding, item-specific information was selectively observed in VVS and hippocampus, but not in the PFC. During maintenance, the geometry of activity patterns in the PFC was captured by the deepest layers of a recurrent DNN model. This effect was transient, locked to the presentation of the cue, and was specific to the beta (16-28Hz) frequency band. Our results suggest that PFC represents task-relevant information in an abstract representational format, dynamically affecting distributed representations in the VVS and hippocampus through top-down inhibitory mechanisms.

Spatial working memory (SWM) is the short-term maintenance and update of spatial information. SWM is important for a broad range of cognitive tasks that involve reasoning about physical objects and spatially-organized information, even in abstract non-spatial contexts. What are the neural computations underlying SWM? We trained rats to perform a novel, parametric SWM task (PSWM), where 2D visual stimuli are projected onto the floor of a large behavioral arena, and animals learn to maintain stimulus locations in their working memory over a delay period. Our projection arena exploits the previously-reported tendency of rodents to attend to stimuli close to the ground, and their rapid acquisition of visual tasks involving such stimuli. Our paradigm additionally overcomes several key limitations of existing rodent SWM tasks: target location is continuous as opposed to discrete, and can be systematically controlled. It also spans a large space within animals’ field of view, and mimics rodent naturalistic behavior. Animals were successfully trained to perform the task over several weeks, and we obtained a precise behavioral readout of the spatial precision of working memory in continuous spatial coordinates. Future experiments will involve the use of multiple tools to monitor and perturb brain activity during task performance, allowing the detailed investigation of the neural computations underlying PSWM behavior.

Do we search information in our minds just as we search in perception? Here we examined one potential difference—the effectiveness of low versus high-level features. In visual search, attention is guided more efficiently by low-level (e.g., color) than high-level features (e.g., identity), reflecting that low-level features are processed earlier in the visual hierarchy. However, some evidence suggests that lower-level features may have slower access for memory search (Kong & Fougnie, 2021). Here we examined how low- vs. high-level features are searched in perception and memory. In two experiments, participants were either pre-cued or retro-cued to localize real-world objects defined by color (color patch) or object identity (Expt. 1 – object label, e.g., car; Expt. 2 – category label, e.g., vehicle). Not surprisingly, visual search for color was faster than search for object or category identity. However, across both studies, we found no evidence that memory search for color was faster than object or category identity. These findings suggest that unlike perception, which is progressively processed in a bottom-up manner, memory representations may be simultaneously accessed at both higher and lower levels of representation.

The number of items in working memory, or WM load, can be decoded using multivariate analysis of EEG voltage. Recent work proposes that these signals may be tracking the operation of a spatiotemporal “pointer system” that enables continuous tracking of stored items in both space and time. The extant work has used displays in which items were distinguished by their spatial positions. Here, we examined whether the same multivariate analysis can decode WM load when items are separated only by their temporal positions. Participants stored two successive arrays of items in WM while EEG activity was recorded. The second array of items appeared in either novel positions or in the same positions as the first array, so that only temporal position separated the items between arrays. We observed robust decoding of the total number of items stored in both conditions, and load-specific patterns of activity were equivalent between conditions. These findings suggest that multivariate decoding of WM load is tapping into the operation of a pointer system in which items can be individuated based on either spatial or temporal position.

The cross-modal effects between auditory and visual cortices are not equivalent (Wolff et al., 2020). This study investigated if long-term associations between visual and auditory stimuli could increase the cross-modal effects. First 6 auditory pure tones were learned in association to 6 visual orientation gratings. Next, a discrimination task for the orientations was completed with simultaneous EEG recording. Orientation memories were recalled either from long-term memory via their auditory counterpart, or were visually presented. The orientation memories were decoded from the EEG responses to task irrelevant auditory and visual impulses presented at memory delay. The results showed that an auditory impulse could perturb visual WM network, when auditory cues were used but not when orientations were visually presented. The neural patterns of orientations generalized across memory conditions confirming that the auditory impulse pinged visual memories. Thus, long-term memories are rapidly ported to WM and the long-term connections drive cross-modal effects.

Working memory (WM) content is mostly stored in neurons preferring contralateral cues in bilateral prefrontal cortex (PFC) [2], but can travel between hemispheres [3]. This is thought to support full-field spatial WM continuity. Recently, temporal continuity in WM has been linked to activity-silent mechanisms in PFC supporting behavioral serial dependence (SD) between successive trials [1]. Moreover, SD increases when prefrontal activity-silent traces of previous memories are reactivated in the fixation period by either internal or external (e.g. transcranial magnetic stimulation, TMS) inputs [1]. How memory traces and reactivations interact with anatomical lateralization to ensure both WM spatial and temporal continuity is currently unknown. Here, we asked if SD is lateralized and how its mechanisms are propagated between hemispheres. We tested the lateralization of SD using human and monkey behavioral responses and TMS experiments in humans, and we analyzed simultaneous bilateral PFC multiunit recordings in one monkey performing a spatial WM task to assess interhemispheric transfer of fixation-period reactivations.

Our surroundings contain rich and complex temporal structures, such as speech and music. Humans are able to quickly adapt when these events unfold at a faster or slower pace. Can humans speed up or slow down their rate of working memory encoding when they expect brief versus long stimuli? In a series of delayed estimation experiments, participants had to encode gabor patches that were presented for different amounts of time. Memory precision was characterized by maximum capacity and rate of encoding. We hypothesized that sequential presentation of brief vs. long stimuli would induce strong expectations of presentation time, which would in turn increase or decrease the rate of encoding. Indeed, we found that the presentation time of the preceding two stimuli selectively affected encoding speed on the current trial. In sum, we show that humans can adaptively speed up or slow down encoding dynamics in working memory.

How are stimuli represented in working memory, and how are these representations translated into decisions? In orientation reconstruction tasks participants are presented with two oriented gratings and asked to reconstruct one of them after a short delay. Previous studies argue that in such paradigms working memory noise arises from drifts in the mean of the representation, rather than increases in the uncertainty. However, it is not known how that representation is then turned into a response. Adapting the same orientation reconstruction task, we show that participants’ responses reveal a distinct pattern of bias: vertical or horizontal gratings are reconstructed accurately, but even the smallest tilts away from the cardinal axes are exaggerated in recall. Thus, the data are not symmetric with respect to exchange of response and sample values. By studying this asymmetry, we are able to distinguish between three competing response rules: responses based on the mean, the mode (point of highest probability) or random samples of a distributional belief. We find that the mean-based response model matches observers’ behaviour closely, while both mode and sample-based strategies would result in systematic deviations from the data. We additionally study the variability of the response distributions with different working memory delay times. We conclude that a representation encoding a 1-D sufficient statistic (e.g. mean value) cannot explain the data, and rather some higher dimensional code is required. This suggests that the internal belief representing memory of continuous values of orientation carries uncertainty, as well as the mean value, over time.

The human eye scans visual information by following a series of fixations forming scan-paths. Analogous to these scan-paths during the process of actual “seeing”, we investigated whether similar scan-paths are observed while subjects are “rehearsing” stimuli in visuospatial short-term memory. Participants performed a continuous recall task in which the precise location and color of serially presented discs were to be rehearsed during retention and later to be reproduced during recall. We varied the direction along which the items were presented and investigated whether scan-paths during rehearsal followed the same scan-paths of encoding (left-to-right/right-to-left). We confirmed the hypothesis that the eyes follow similar scan-paths during encoding and rehearsal, and found that these retrospective scan-paths are associated to recall precision. Our findings support an attention-based rehearsal mechanism whereby eye movements not only indicate where attention is focused, but also show how spatial attention is shifted during maintenance.

Recent developments in the field of working memory (WM) have proposed that working memory contents are stored in two complementary storage systems within the brain. Active states, which are identifiable by electroencephalogram (EEG) and latent states which cannot be measured by EEG. While this distinction in states has been consistently found across studies, little is known about the functionality of these different states. In the current set of experiments, we are aiming to characterise the functionality of different WM states. In a first experiment, participants were asked participants to remember two memory items – a gabor patch or a colour – and systematically interfered with either the cued or uncued item by matching the feature dimension of a distractor task in the middle of each trial. Initial results show that error rate is reduced if the interference task is not matching on the feature dimension of the active items. This gives a first indication that active states might be more prone to interference.

Contraction bias is a phenomenon where the judgment of the magnitude of items held in working memory is biased towards the average of past observations. This phenomenon has been first described more than a century ago and since then, has been replicated in various decision making tasks in humans and rodents. Contraction bias is assumed to be an optimal strategy by the brain, given the noisy nature of working memory. From a Bayesian perspective, the progressive shift of the noisy memory towards the mean of a prior distribution built from past sensory experience helps with more accurate estimates of the memory. In this work, we propose an alternative mechanism, via short-term history biases, or serial dependence. Our model is motivated and inspired by recent results from an auditory delayed-discrimination task in rats, where the posterior parietal cortex (PPC) has been shown to be critical to these memory effects. The dynamics of our model suggests that contraction bias can emerge as a result of a volatile working memory content which makes it susceptible to shifting to the past sensory experience. The errors, at the level of individual trials, are sampled from the full distribution of the stimuli, and are not due to a gradual shift of the memory towards the distribution’s mean. Our model explains both short-term history biases or serial dependence, as well as contraction bias towards mean for the averaged performance. The results are consistent with the role of the PPC in encoding such sensory history biases, and provide predictions of performance across different stimulus distributions and timings, delay intervals, as well as specific neuronal dynamics in putative working memory areas.

Multivariate load analyses of EEG track content-independent pointers, rather than specific featural information. In a change detection task, participants remembered one feature from an array of color-orientation conjunction items. The attended feature can be decoded, suggesting there is a detectable difference between the conditions. However, when training a load model on one feature, there is little significant loss in accuracy when testing on the other feature. This method can also evaluate the degree to which distractors are encoded into working memory. In a whole report task, participants remembered 1 or 2 digits. In distractor conditions, participants had to ignore a hashtag (#) or letter which appeared simultaneously with the digits. Multivariate analyses revealed that the hashtag is better ignored than the letter distractors. Additionally, in the final 2 blocks, participants had to ignore the digits and remember the letters. This led to encoding of the digit distractor, suggesting a selection history effect.

Although many studies point to qualitative similarities between working memory (WM) and attention, the degree to which these two constructs rely on common neural substrates remains unclear. In this study, we used fMRI to compare the patterns of neural activity evoked during attentional selection and selection within WM. Eleven participants selected one of three visual objects either while that information was present on the screen (i.e., attentional selection) or after it disappeared (i.e., WM selection ). Our measure of selection was how well classifiers trained on the pattern of BOLD activation could decode the location of the selected object. Critically, the classifiers were trained and tested either in the same condition (e.g., trained and tested on attentional selection; within-condition decoding) or different conditions (e.g., trained on attentional selection and tested on WM selection; across-condition decoding). Comparing the within- and across- condition performance provides quantitative evidence for the amount of overlap in neural substrates between the selection process in WM and attention. We observed significantly above-chance decoding performance both within- and across-conditions in visual cortex and intraparietal sulcus. The time-series of decoding performance within- and across-conditions was consistent with the time-course of the selection process, confirming the classifiers decoded the selection process itself. Importantly, the above-chance within- and across- condition performance was largely comparable in these areas—the ratio of across-condition to within-condition performance is larger than 0.91. These results point to a strikingly high overlap in the neural representations between attentional and WM selection.