Simple Cycle Reservoirs are Universal

• DOI: 10.48550/arxiv.2308.10793
• Publication Date: 2023-01-01
• Authors: Boyu Li; Robert Simon Fong; Peter Tiňo
• Keywords: Reservoir computing

Reservoir computation models form a subclass of recurrent neural networks with fixed non-trainable input and dynamic coupling weights. Only the static readout from state space (reservoir) is trainable, thus avoiding known problems propagation gradient information backwards through time. have been successfully applied in variety tasks were shown to be universal approximators time-invariant fading memory filters under various settings. Simple cycle reservoirs (SCR) suggested as severely restricted reservoir architecture, equal weight ring connectivity units input-to-reservoir weights binary nature same absolute value. Such architectures are well suited for hardware implementations without performance degradation many practical tasks. In this contribution, we rigorously study expressive power SCR complex domain show that they capable approximation any unrestricted linear system (with continuous readout) hence filter over uniformly bounded streams.