The restricted isometry property for Echo state networks with applications to sequence memory capacity

Han Lun Yap, Adam S. Charles, Christopher J. Rozell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The ability of networked systems (including artificial or biological neuronal networks) to perform complex data processing tasks relies in part on their ability to encode signals from the recent past in the current network state. Here we use Compressed Sensing tools to study the ability of a particular network architecture (Echo State Networks) to stably store long input sequences. In particular, we show that such networks satisfy the Restricted Isometry Property when the input sequences are compressible in certain bases and when the number of nodes scale linearly with the sparsity of the input sequence and logarithmically with its dimension. Thus, the memory capacity of these networks depends on the input sequence statistics, and can (sometimes greatly) exceed the number of nodes in the network. Furthermore, input sequences can be robustly recovered from the instantaneous network state using a tractable optimization program (also implementable in a network architecture).

Original languageEnglish (US)
Title of host publication2012 IEEE Statistical Signal Processing Workshop, SSP 2012
Pages580-583
Number of pages4
DOIs
StatePublished - 2012
Externally publishedYes
Event2012 IEEE Statistical Signal Processing Workshop, SSP 2012 - Ann Arbor, MI, United States
Duration: Aug 5 2012Aug 8 2012

Publication series

Name2012 IEEE Statistical Signal Processing Workshop, SSP 2012

Conference

Conference2012 IEEE Statistical Signal Processing Workshop, SSP 2012
Country/TerritoryUnited States
CityAnn Arbor, MI
Period8/5/128/8/12

Keywords

  • Compressed Sensing
  • Echo State Networks
  • Sequence Memory

ASJC Scopus subject areas

  • Signal Processing

Fingerprint

Dive into the research topics of 'The restricted isometry property for Echo state networks with applications to sequence memory capacity'. Together they form a unique fingerprint.

Cite this