Diagnostic Pathology: Open Access
Open Access

In a recent paper it was reported from the Betula project that fragmented cues consisting of ”well-consolidated face-name memory traces” used in tests for pattern completion were “found to be a sensitive marker of age-related cognitive deficits”. The Swedish Betula project has “one of the longest existing longitudinal datasets on neurocognitive function” [1,2].

The ability to complete patterns with fragmented or to correct patterns with distorted cues is related to age-related episodic memory deficits [3,4]. One of the characteristics of Alzheimer’s disease is episodic memory deficits and therefore memory correction or completion is relevant for the diagnosis of Alzheimer’s disease.

Alzheimer’s disease, first diagnosed as presenile dementia by the German psychiatrist Alois Alzheimer in 1906, is a debilitating neurological disease that presently has no cure, only medications that can moderate symptoms of the disease are available [5-7]

The beginning of the disease is inconspicuous the nervous system is able to compensate for a considerable level of neurological damage before symptoms appear. In anticipation of a possible cure it is therefore of interest to find tests that facilitate early diagnosis of the disease [8].

Efforts at mathematical modelling of neurons and their connections by synapses as networks started as early by McCulloch and Pitts neuron model [9], inspired by digital circuitry. The progress of such endeavors has been considerable, resulting in e.g. neuromorphic electronic systems, pioneered by Mead [10].

Associative memories are a subset of this field. One notable such circuit was described by Little, further developed by Hopfield networks. The networks can store a number of patterns and have been applied for e.g. image detection and recognition. The memories to be stored in the original Hopfield network were vectors with binary (0,1) or bipolar (-1,1) numbers. The outputs were vectors of the same kind and the same dimension. The Hopfield network has been developed further, even recently [11-13], but it is the original network that is employed in this paper.

It is here of interest that Hopfield networks have been applied in efforts to simulate the progression of Alzheimer’s disease [14-19]. Connection elements, “synapses” are eliminated and the decline of the network’s ability to recognize stored patterns is studied. The networks display “graceful degradation”, i.e. they perform well with a number of connective elements eliminated. When degradation of performance sets in, the decline becomes rapid and performance itself becomes eliminated. In all these papers it is the ability of the degraded neural network to correctly establish the learned memory that is studied. This is in contrast to the present paper in which the process of correctly establishing the learned memory is the subject of study.

It will be shown in this paper that, in spite of this graceful degradation it is possible to determine by the network’s performance that connection elements have been eliminated at very small relative numbers it is just a matter of a judicious choice of presented patterns. This leads up to the hypothesis that a similar choice of patterns shown to humans could make early diagnosis of Alzheimer’s disease possible.

In Figure 1 we show five patterns, or logos, by the Swedish artist Tex Berg, in the upper row. These logos, described by 153 element vectors determined both by the positions of the triangles’ vertices and their colours, have been entered into the Hopfield network as memories.

In the middle row we show the same logos slightly distorted, lowlevel noise has been added to the vectors that represent the undistorted logos. These have subsequently been entered into the network as keys or initial states in search of their memories.

In the bottom row we see the final outputs from the Hopfield network. All keys were recognized and restored to their respective originals. The text under the second row tells us that seventeen per cent of the connecting elements had been eliminated in this experiment. The fact that still all logos were recognized shows that this Hopfield network has a very large overcapacity. This Hopfield network could contain over 20 different logos without interferences [20].

An important observation can be made in Figure 1. The interesting data here are the numbers 2, 2, 1, 2, 1 below the logos in the third row. In order to restore each logo, the key must be input to the weight matrix of the Hopfield network. A minimum of 1 time is needed. If this does not succeed in restoring the original logo the intermediate result will be input again to the weight matrix. Here this happened for the first, second and fourth keys. The point to be made is that there was a delay for the first, second and fourth keys in the process of recognizing the logos. It is this delay that tells us that this Hopfield network has suffered elimination of connection elements. In many simulations we see a number 3, which means that the process has been repeated once more, the process is iterative. In the following we count it as delay if the iterative process has started (i.e. the number is >1). Let us look at some statistics.

Figure 1: The three rows show the remembered patterns, or logos, (top row), the distorted logos used as keys (middle row) and the logos recognized and restored by the Hopfield network (bottom row).

The results from many simulations sixty simulations each with five logos for six levels of elimination of connection elements and for two levels of distortion (low distortion and moderate distortion) make it possible to draw an important conclusion: The delay needed to recognize the distorted logo grows with the number of eliminated connection elements.

It is important to note that no information can be derived from the case where the keys are undistorted logos. It is only with distorted logos that the statistics show us that connecting elements have been eliminated. It may be noted that delays appear and become overwhelming even when the keys are undistorted but first at a much higher number of eliminated connection elements than shown in Figure 2. Let us also look at the success rate of recognizing logos.

Figure 2: Statistical results showing relative number of delays vs. relative number of connection elements, or synapses, eliminated.

We notice that all three curves have one thing in common there is no trend for reduced success rate in restoring the original logo. The success rate is not a measure of the elimination of connection elements. As with the time delays, reduction in success rates appears and they drastically decline even when the keys are undistorted but first at a much higher number of eliminated connection elements than shown in Figure 3.

Figure 3: Statistical results showing success rates vs. relative number of connection elements, or synapses, eliminated.

We see that when the logos are undistorted (solid curve) all logos are recognized at all numbers of elimination of connection elements. In the case where the distortion is low (dotted curve) almost all logos are recognized. In the third case where the distortion is moderate (dashed curve) a larger number of “efforts” by the Hopfield network did not end in a successful recognition, but again that number does not show any interesting dependency on the number of eliminated connection elements.

The conclusion is: when distorted logos are shown as keys, the number of delays necessary to reach a successful recognition is the only measure that indicates the number of eliminated connection elements (as long as this number is relatively small).

The conclusion can also be phrased as:

1. At small numbers of eliminated connection elements the learned logo is perfectly preserved. This is obvious by the fact that the undistorted logo is immediately identified as a learned logo, without an iterative process for identification.

2. At small numbers of eliminated connection elements an iterative process is necessary if the presented logo is distorted. And the demand for an iterative process increases if the distortion is increased.

It is obvious that in the case of “artificial Alzheimer’s disease” the tested stimuli should resemble but not coincide with the learned memories to force an iterative process. Based on the above simulation results we will now propose a hypothetical early diagnostic test for Alzheimer’s disease. We need some kind of patterns that are commonly known, i.e. memorized, to humans or at least a great number of humans. Here we limit ourselves to literate people of the Western world. They have all learned some version of the Latin alphabet and the letters are easy to distort. Presented with a series of undistorted letters (not in alphabetical order) and then a series of distorted letters (not in the same order as the undistorted ones) people with eliminated synapses, i.e. beginning Alzheimer’s disease, would take longer time reading the distorted letters out correctly than the undistorted letters, presumably more so than people with no affliction of Alzheimer’s disease. Again, it should be stressed that the results from the simulations, as shown in Figure 3, predict that it is the time to reach the correct result that is dependent on the beginning loss of synapses, not the end result in it. A suggestion for letters for such a test is shown in Figure 4 (the order is alphabetical, though).

Figure 4: The top row shows undistorted letters, the middle and bottom rows show the same letters with different degrees of distortion.

The result of this test can be expected to depend on the status of the test person’s Visual Word Form Area (VWFA), the area in the left fusiform gyrus where letters are recognized [21]. It might also be interesting to study performance on a higher cognitive level. Texts presented with undistorted letters and similar texts with distorted letters would serve that purpose. It is straightforward to replace the Latin alphabet with other alphabets such as the Greek, Cyrillic, and Arabic etc. alphabets. For test persons literate in e.g. Chinese characters, further work is needed.

It was stated that fragmented cues consisting of “well-consolidated face-name memory traces" used in tests for pattern completion were “found to be a sensitive marker of age-related cognitive deficits” [1]. This leads us to an alternative suggestion for a diagnostic test. Hopfield memories are traditionally used also for the purpose of pattern completion. An example is given in where an incomplete rendition of a capital letter ‘T’ needed many iterative loops to be completely restored [22,23].

In pattern completion of fragmented cues the Hopfield memory with deleted connection elements performs just as in the case with distorted cues: the time delays appear long before failures to converge to the correct undistorted pattern. It is thus straightforward to suggest an alternative version of the capital letters, a fragmented version. This is shown in Figure 5. The degree of fragmentation can readily be increased or decreased.

Figure 5: The figure shows fragmented capital letters in random order.

It has been shown that a loss of connection elements in a Hopfield network, a model of associate memory, can be detected in the network’s performance at a very early stage, a stage where the network performs perfectly well, just with some delay. The choice of inputs to the network is all-important: they should be distorted, noisy or fragmented versions of the patterns the network has learned. The inputs must enforce an iterative search process which is not present when the input is a perfect copy of a learned stimulus.

It is therefore proposed that a distorted or fragmented version of letters of the alphabet, patterns that many people have learned well, could serve as an early diagnosis for Alzheimer’s Disease. Is it worthwhile to consider an alternative to the fragmented face test, which, after all, has been proven to be predictive of “age-related cognitive deficits”? A reasonable answer would be that it would be more readily ascertained that a test person is literate and that the letters thus have been learned than to assure that a test person is tested with “a well-consolidated face-name memory”, which would be an individual requirement.

Alzheimer’s disease is characterized by many symptoms. The proposed test presented here doesn’t address all these symptoms, i.e. the test should, if found to be effective, be expected to be sensitive to, but not necessarily specific to Alzheimer’s disease. It could then suitably be a part of the several existing screening tests for early detection of Alzheimer’s disease.

I have met with much encouragement for this effort from my colleagues, professors J. Delsing, M. Liwicki, F. Sandin, K. Zingmark and senior lecturer F. Liwicki.

The research presented here has not necessitated any external funding.

1. Nyberg L, Grande X, Andresson M, Berron D, Lundquist A, et al. (2020) Forecasting memory function in aging: pattern completion ability and hippocampal activity relate to visuospatial functioning over 25 years. Neurobiol Aging 94:217-26.

   [Crossref] [Google Scholar] [PubMed]

2. Nilsson LG, Adolfsson B, Bäckman L, de Frias CM, Molander B, et al. (2004) Betula: a prospective cohort study on memory, health and aging. Aging Neuropsychol. Cogn 11:134-148.

   [Crossref] [Google Scholar] [PubMed]

3. Paleja M, Spaniol J (2013) Spatial pattern completion deficits in older adults. Front. Aging NeuroSci 5:1-6.

   [Crossref] [Google Scholar] [PubMed]

4. Vieweg P, Stangl M, Howard IR, Wolbers T (2015) Changes in pattern completion a key mechanism to explain age-related recognition memory deficits? Cortex 64:343-351.

   [Crossref] [Google Scholar] [PubMed]

5. Alzheimer A (1906) Über einen eigenartigen schweren ErkrankungsprozeB der Hirnrinde. Neurologisches Centralblatt 23:1129-1136.

   [Google Scholar]

6. Breijyeh Z, Karaman R (2020) Comprehensive review on Alzheimer’s disease: causes and treatment. Molecules 25:5789.

   [Crossref] [Google Scholar] [PubMed] 

7. Cummings J, Lee G, Nahed P, Kambar MEZN, Zhong K, et al. (2022) Alzheimer’s disease drug devopment pipeline:2022. TRCI 8:12295.

   [Crossref] [Google Scholar] [PubMed]

8. Rasmussen J, Langerman H (2019) Alzheimer’s disease why we need early diagnosis. Degener Neurol Neuromuscul Dis 9:123.

   [Crossref] [Google Scholar] [PubMed]

9. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull math biophys 5:115-133.

   [Crossref] [Google Scholar] [PubMed]

10. Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78:1629-1636.

    [Crossref] [Google Scholar]

11. Little WA (1974) The existence of persistent states in the brain. Math Biosci 19:101-120.

    [Crossref] [Google Scholar]

12. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. PNAS 79:2554-2558.

    [Crossref] [Google Scholar] [PubMed]

13. Krotov D, Hopfield J (2016) Dense associative memory for pattern recognition. NeurIPS 29:1172-1180.

    [Google Scholar]

14. Horn D, Ruppin E, Usher M, Herrmann M (1993) Neural network modelling of memory deterioration in Alzheimer ’s disease. Neural Comput 5:736-749.

    [Crossref] [Google Scholar]

15. Duch W (2000) Therapeutic implications of computer models of brain activity for alzheimer disease. J Med Inf Technol 5:27-34. [Crossref]

    [Google Scholar]

16. Savioz A, Leuba G, Vallet PG, Walzer C (2009) Contribution of neural networks to Alzheimer disease’s progression. BRB 80:309-314.

    [Crossref] [Google Scholar] [PubMed]

17. Weber M, Maia PD, Kutz JN (2017) Estimating memory deterioration rates following neurodegeneration and traumatic brain injuries in a Hopfield network model. Front Neurosci 11:623.

    [Crossref] [Google Scholar] [PubMed]

18. Pinheiro RF, Colón D, Reinoso LLR (2020) Relating Lurie’s problem, Hopfield’s network and Alzheimer’s disease. CBA 2:1-8.

    [Crossref] [Google Scholar]

19. Aguilar R, Chen H, Deshpande R, Katchur N, Khang M, (2015) et al. Computational modelling of the memory capacity of neural networks and the effects of random neuron failure. Governor's School-library. 1-17.

    [Google Scholar]

20. Haykin S (1998) Neural networks: a comprehensive foundation. Prentice Hall PTR.
21. Dehaene S, Cohen L (2011) The unique role of the visual word form area in reading. TiCS 15:254-62.

    [Crossref] [Google Scholar] [PubMed]

22. Gurney K (1997) An Introduction to Neural Networks. 1st edition, Routledge & CRC Press, London, UK.
23. De Book EE, De Deyn PP, Dierckx E, Engelborghs S (2019) Brief cognitive screening instruments for early detection of Alzheimer’s disease: a systematic review. Alzheimer's Res Ther 11:1-.

    [Crossref] [Google Scholar] [PubMed]