Learning and Hopfield Networks
Among the prominent types of neural networks studied by cognitive scientists, Hopfield
networks most closely model the high-degree of interconnectedness in neurons of the
human cortex. The papers by McClellan et al. (1995) and Maurer (2005) discuss
learning systems in the human brain-mind system and the role of Hopfield networks as
models for actual human learning including sequence of items learned. This resonates
with Chomsky’s emphasis on the role of syntax and word sequence in language
learning.
Write a paper in which you argue for or against the notion that Hopfield networks not
only share characteristics of learning in the brain-mind system but also that they are
useful analytic tools.
Learning and Hopfield Networks
Introduction
Learning involves the formation patterns of neural wiring that are very useful irrespective of
presence or absence of external feedback from the supervisor. For instance, there are neural
wiring patterns in both absence and presence of external feedback from the environment or an
instructor. Thus, neural networks (both artificial and biological) that facilitate learning both in
supervised and unsupervised means are usually very essential in enabling extraction of useful
relationships from input/sensory data. However, the patterns extracted may be invariant
patterns in key variants and the data that allow differentiation of important input classes in
the system (McClellan et al., 1995). Either way, these concepts are learned by the system on
itself. Therefore, learning systems in the human brain-mind system takes place through the
biological neural network and the Hopfield networks as models plays a very important role
for actual human learning where the sequence of items learned is also included (Hopfield,
1982). The Hopfield network resonates with the emphasis of Chomsky on the role of word
sequence and syntax in the process of learning language (Chomsky, 2009).
Hopfield network is one of Artificial Neural Networks (ANN) which is involved in
processing of information paradigm whose inspiration originates from the by the way in
LEARNING AND HOPFIELD NETWORKS 2
which processing of information takes place in the brain (Squire & Kandel, 1999). ANNs are
mathematical models involved in the emulation of some of properties of biological nervous
systems observed and drawn on the adaptive biological learning analogies. This implies that
Hopfield network is not only concerned with emulation of the learning systems in the human
brain system, but also an analytical tool (McClellan et al., 1995). Therefore, Hopfield
network as an ANN ii is usually composed of an extensive number of processing elements
that are highly interconnected that are analogous to biological neurons and interconnected
together by connections that are analogous to synapses (Burgess & O’Keefe, 1994). However,
Hopfield networks not only share the brain-mind system learning characteristics, but they are
also important analytic tools.
The Hopfield Network as an Analytical Tool
The Hopfield network has the possibility of acting as an analytical tool since it is represented
as nodes in the network that represents extensive simplifications of real neurons, and they
usually exist in either firing state or not firing state (Hopfield, 1982). In addition, all the
nodes in a Hopfield network are connected to each other with some strength which allows it
to execute its functions in an effective manner. Apart from the Hopfield network deriving
meaning from sets or subsets of data for the purpose of facilitating learning process, it also
make sure that relationships from the same sets or subsets of data which means they provides
an aspect of data analytics (Squire & Kandel, 1999). This implies that apart from Hopfield
network playing a crucial in facilitating brain-mind system learning, it is also a very useful
analytical tool which through the relationships it derives from data serves as a way of
ensuring the meaning of data is achieved. As an effective analytical tool, Hopfield network at
any time will make sure that the interconnected nodes will change their state (i.e. stop or start
firing) on the basis of received inputs from the other nodes (McClellan et al., 1995).
LEARNING AND HOPFIELD NETWORKS 3
This implies that Hopfield can also be used as a powerful analytical tool to create
relationships between received inputs or solve combinatorial problem (Hopfield, 1982).
Furthermore, the analytical tool aspect of Hopfield involves certifying the accuracy of
algorithm. Rizzuto and Kahana (2001) showed that the neural network model, especially the
Hopfield network model is essential in the process of accounting for repetition on recall
accuracy through incorporation of an algorithm that is based on probabilistic-learning and
analysis. It is also reiterated that during the retrieval process which is common during
analysis, no learning occurs thereby making the network weights to remain fixed in addition
to showing the capability of the Hopfield network model to switch to a recall stage from a
learning stage (McClellan et al., 1995).
Moreover, in the attempts of using Hopfield network model as an analytical tool,
combinatorial optimization it highly prioritized especially through correct modelling of the
problem (Hopfield, 1982). Therefore, the Hopfield network model has the ability to give
some solution and some minimum but in most cases it is not able to find the optimal solution
to a problem even though the analysis provided is often sufficient. This implies that Hopfield
network is generally used as a “black box” for the calculation of some output that result from
a particular self-organization because of the network (Burgess & O’Keefe, 1994). Therefore,
due to the ability of Hopfield network model to be used as an analytical tool, it is then
possible to use it for combinatorial problem. This is mainly because, when a problem is
writable in the energy function of an isomorphic form, then it is possible to find the
function’s local minimal using the Hopfield network model, but there is no guarantee that the
solution provided will be optimal (Squire & Kandel, 1999).
For example, the use of Hopfield network model as an analytical tool has been applied in the
travelling salesman problem. This is mainly because this is an NP-hard problem in the
analyses involved in combinatorial optimization (Burgess & O’Keefe, 1994). For instance,
LEARNING AND HOPFIELD NETWORKS 4
since travelling salesmen travel extensively then, given a list of cities alongside their
distances in pair, the Hopfield network model can be used to find a tour that will be shortest
of all others, but enables each city to be visited by the salesman (Rolls & Treves, 1998). In
this scenario, the tour path passes through n number of cities, and each of those cities should
only be visited by the salesman once prior to returning to his or her original point of
departure as well as ensuring that the distance covered is minimal. However, Hopfield
network has been extensively used to resolve this combinatorial problem after it has been
expressed in terms of energy function, but absolute optimality is not always guaranteed (Rolls
& Treves, 1998). Irrespective of this shortcoming, Hopfield network model remains a very
useful analytical tool since when unit is regarded as a small processor, Hopfield neural
network can be used to offer high power of parallelism and computation. This is achievable
because of the inherent analytical capability of the Hopfield network which can undergo
asynchronous updating (Squire & Kandel, 1999).
Conclusion
Since the Hopfield network was published in 1982 by John Hopfield, it became a
breakthrough in neural network as a result of providing significant dynamism to the research
on neural networks, particularly on their analytical capacity. Much has so far been discovered
and these findings are nowadays used to refine the analytical capabilities of artificial neural
systems, especially in the field of computer science.
LEARNING AND HOPFIELD NETWORKS 5
References
Burgess, M. R. N. & O’Keefe, J. (1994). A model of hippocampal function, Neural Networks,
7, pp. 1065-1083.
Chomsky, N. (2009). Mysteries of Nature: How Deeply Hidden? Journal of Philosophy, 106
(4): 167–200.
Hopfield, J. (1982). Neural networks and physical systems with emergent collective
computational abilities, Proceedings of the National Academy of Sciences, 79, pp.
554-2558.
McClellan, J., McNaughton, B. & O’Reilly, R.C. (1995). Why there are complementary
learning systems in the hippocampus and neocortex: Insights from the successes and
failures of connectionist models of learning and memory, Tech. Rep. PDP.CNS.94.1,
Carnegia Mellon University. March, 1994.
O’Reilly, R. C. & Munakata, Y. (2000). Computational Explorations in Cognitive
Neuroscience. Cambridge, MA: The MIT Press.
Rizzuto, D. S. & Kahana, M. J. (2001). An auto-associative neural network model of paired-
associate learning. Neural Computation, 13, 2075-2092.
Rolls, E. & Treves, A. (1998). Neural Networks and Brain Function. New York, NY: Oxford
University Press.
Squire, L. & Kandel, E. (1999). Memory: From Mind to Molecules. New York, NY: Henry
Holt and Company.