August 2010
Volume 10, Issue 7
Free
Vision Sciences Society Annual Meeting Abstract  |   August 2010
Promoting generalization by hindering policy learning
Author Affiliations
  • Jacqueline M. Fulvio
    Department of Psychology, University of Minnesota
    Center for Cognitive Sciences, University of Minnesota
  • C. Shawn Green
    Department of Psychology, University of Minnesota
    Center for Cognitive Sciences, University of Minnesota
  • Paul R. Schrater
    Department of Psychology, University of Minnesota
    Center for Cognitive Sciences, University of Minnesota
    Department of Computer Science, University of Minnesota
Journal of Vision August 2010, Vol.10, 1142. doi:10.1167/10.7.1142
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Jacqueline M. Fulvio, C. Shawn Green, Paul R. Schrater; Promoting generalization by hindering policy learning. Journal of Vision 2010;10(7):1142. doi: 10.1167/10.7.1142.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

A pervasive question in perceptual and motor learning concerns the conditions under which learning transfers. In reinforcement learning, an agent can learn either a policy (i.e., a mapping between states and actions) or a predictive model of future outcomes from which the policy can be computed online. The former is computationally less expensive, but is highly specific to the given task/goal, while the latter is computationally more expensive, but allows the agent to know the proper actions to take even for novel goals. Policy learning is appropriate when forward look ahead is not required and the number of policies to be learned is small, while model learning is appropriate under the opposite conditions. Therefore, by manipulating these factors in a given task, the degree of transfer should be predictably altered as well. The current study tests this hypothesis with a navigation task requiring subjects to steer an object through a novel flow field to reach visible targets as quickly as possible. We vary the predictive component of the task by manipulating the amount of control subjects have over the object. Half steer the object for the entire duration of the experiment, which favors policy learning, while the rest lose control intermittently, which adds a look ahead component to the task and favors model learning. We vary the number of policies to be learned by manipulating the number of target locations the subject reaches where large numbers are expected to favor model learning. Half have only two target locations to reach while the rest have twelve. Performance on transfer tasks where the environment is held constant, but the goal is altered, is better for those subjects trained under conditions that favor model learning. These results suggest that developing training tasks that discourage simple policy learning is critical if generalization is desired.

Fulvio, J. M. Green, C. S. Schrater, P. R. (2010). Promoting generalization by hindering policy learning [Abstract]. Journal of Vision, 10(7):1142, 1142a, http://www.journalofvision.org/content/10/7/1142, doi:10.1167/10.7.1142. [CrossRef]
Footnotes
 ONR N 00014-07-1-0937.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×