Structure in the space of value functions

David Foster, Peter Dayan

Research output: Contribution to journalArticlepeer-review

34 Scopus citations


Solving in an efficient manner many different optimal control tasks within the same underlying environment requires decomposing the environment into its computationally elemental fragments. We suggest how to find fragmentations using unsupervised, mixtures model, learning methods on data derived from optimal value functions for multiple tasks, and show that these fragmentations are in accord with observable structures in the environments. Further, we present evidence that such fragments can be of use in a practical reinforcement learning context, by facilitating online, actor-critic learning of multiple goals MDPs.

Original languageEnglish (US)
Pages (from-to)325-346
Number of pages22
JournalMachine Learning
Issue number2-3
StatePublished - Nov 2002
Externally publishedYes


  • Density estimation
  • Dynamic programming
  • Mixture models
  • Reinforcement learning
  • Unsupervised learning
  • Value functions

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Structure in the space of value functions'. Together they form a unique fingerprint.

Cite this