September 2021
Volume 21, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2021
Evolving visual representations from noise
Author Affiliations & Notes
  • Guido Maiello
    Justus Liebig University Giessen
  • Katherine R Storrs
    Justus Liebig University Giessen
  • Alexandra Quintus
    Justus Liebig University Giessen
  • Roland W Fleming
    Justus Liebig University Giessen
  • Footnotes
    Acknowledgements  This research was supported by the DFG (IRTG-1901: ‘The Brain in Action’, SFB-TRR-135: ‘Cardinal Mechanisms of Perception’), and an ERC Consolidator Award (ERC-2015-CoG-682859: ‘SHAPE’). Author KRS was supported by an Alexander von Humboldt fellowship.
Journal of Vision September 2021, Vol.21, 2544. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Guido Maiello, Katherine R Storrs, Alexandra Quintus, Roland W Fleming; Evolving visual representations from noise. Journal of Vision 2021;21(9):2544.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Seeing involves not only the light that reaches our retinae, but also visual memories and expectations. When searching for our favourite mug, we hold its appearance in mind. When talking to a friend over the phone, we picture their facial expressions. Talented artists can even paint detailed visual scenes from memory. The content of such mental representations is of great interest in perceptual and cognitive neuroscience, yet is challenging to measure experimentally (alas, too few of our participants are talented artists!). One previously-proposed approach is reverse correlation. Participants are asked to report whether they see a signal—e.g., the letter “s”—in pure noise. Over enough trials, some random noise samples will happen to resemble the signal the participant has in mind. Averaging over all trials in which a participant spuriously detected a signal provides a “classification image” — a visualisation of their representation of the letter. One major drawback of reverse correlation is that it requires tens of thousands of trials to obtain even coarse impressions of mental images. Perhaps more importantly, reverse correlation cannot recover multiple concurrent templates. If a participant imagined both print and cursive instances of the letter “s”, the recovered classification image would be a jumble of both. We address these issues using an evolutionary algorithm approach. We generate image populations by crossbreeding noise. On each trial, participants are shown multiple alternative images drawn from these populations. Across generations, only those images in which participants detect a signal are kept for further breeding. In both simulation and experiments with human participants, we demonstrate that this method converges faster than standard reverse correlation, can recover multiple internal representations of a signal, and can even provide access to the mental representation of illusory visual percepts. Our approach thus provides an efficient, data-driven way to access complex mental representations.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.