October 2003
Volume 3, Issue 9
Vision Sciences Society Annual Meeting Abstract  |   October 2003
Shape recipes: scene representations that refer to the image
Author Affiliations
  • William T Freeman
    Massachusetts Institute of Technology, Artificial Intelligence Laboratory, USA
Journal of Vision October 2003, Vol.3, 419. doi:https://doi.org/10.1167/3.9.419
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      William T Freeman, Antonio Torralba; Shape recipes: scene representations that refer to the image. Journal of Vision 2003;3(9):419. https://doi.org/10.1167/3.9.419.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The goal of low-level vision is to estimate an underlying scene (e.g., shape or reflectance) from an observed image. Real-world images and scenes can be very complex, conventionally requiring high dimensional representations that are difficult both to estimate and to store. We propose a low-dimensional representation, called a scene recipe that relies on the image itself to describe the complex scene configurations. The scene recipe is a formula telling how to transform the local image information to the desired scene quantities. In many situations, scene and image are closely related and it is possible to find such a functional relationship.

Shape recipes are an example: these are the regression coefficients that predict the bandpassed shape from local image data (in some cases, after a point non-linearity). This representation can have appealing properties such as slow variation over space and scale. We show how to exploit the slow variation over scale by first learning the recipes relating image to shape at low spatial frequencies, then applying those recipes at high spatial frequencies to infer high resolution shape, improving the initial shape estimate. Shape recipes implicitly contain information about lighting and materials and they may also be useful for material segmentation.

These scene representations always require that the image be available in order for the scene recipe to compute the shape or other scene quantity. In that sense, they are consistent with theories that suggest that the visual system uses the world as a visual memory, not storing in the brain what can be obtained by looking.

Freeman, W. T., Torralba, A.(2003). Shape recipes: scene representations that refer to the image [Abstract]. Journal of Vision, 3( 9): 419, 419a, http://journalofvision.org/3/9/419/, doi:10.1167/3.9.419. [CrossRef]

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.