Neural network architectures for temporal pattern processing
I present a general taxonomy of neural net architectures for processing
time-varying patterns. This taxonomy subsumes many existing architectures in
the literature, and points to several promising architectures that have yet
to be examined. Any architecture that processes time-varying patterns
requires two conceptually distinct components: a short-term memory that holds
on to relevant past events and an associator that uses the short-term memory
to classify or predict. My taxonomy is based on a characterization of
short-term memory models along the dimensions of form, content, and
adaptability. Experiments on predicting future values of a financial time
series (US dollar-Swiss franc exchange rates) are presented using several
alternative memory models. The results of these experiments serve as a
baseline against which more sophisticated architectures can be compared.
Retrieve Paper
(pdf of text without figures)
(pdf of figures)