# Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Memristor Models

**Assumption**

**1.**

**Assumption**

**2.**

## 3. Memristor Neural Network Models

#### 3.1. Ideal Memristor Neural Network Model

#### 3.2. Real Memristor Neural Network Model

## 4. Invariants of Motion and Lyapunov Functions

**Property**

**1.**

**Proof.**

**Property**

**2.**

**Proof.**

## 5. RMNNs in the Flux–Charge Domain

**Property**

**3.**

**Proof.**

**Property**

**4.**

**Proof.**

**Property**

**5.**

**Proof.**

## 6. Main Results on the Convergence of RMNNs

**Property**

**6.**

**Proof.**

**Theorem**

**1.**

- (1)
- $\varphi (t;{\phi}_{0})$ converges to $\eta \in {\mathbb{R}}^{n}$ as $t\to \infty $, where η is an EP of (18);
- (2)
- $(v(t;{v}_{0},{\phi}_{0}),\phi (t;{v}_{0},{\phi}_{0}))$ converges to the EP $(0,\eta )\in {\mathbb{R}}^{2n}$ of (7) as $t\to \infty $;
- (3)
- $(\psi (t;{\phi}_{0},0),{\mathcal{Q}}_{\mathrm{R}}(t;{\phi}_{0},0))$ converges to the EP $(\eta ,{Q}_{\mathrm{R}}^{\infty}({v}_{0},{\phi}_{0}))$ of (16) as $t\to \infty $.

- Theorem 1 can be considered an extension of the convergence results obtained in [32] for IMNNs to NNs with real memristors. Basically, Theorem 1 states that the presence of rectifying nonlinear resistors in the neuron model does not destroy the property of convergence that holds for symmetric IMNNs.
- It is worth remarking that the assumption of isolated EPs for the asymptotic system (18) is not restrictive. Indeed, in the case in which the system has non-isolated EPs, the vector field defining (18) can be changed by an arbitrarily small amount to obtain isolated EPs. This can be shown via an argument based on the Sard theorem analogous to that used to prove ([32], Property 2) (details are omitted).
- The convergence result in Theorem 1 has been proven via a Lyapunov approach applied to the system describing the RMNN in the flux–charge (integral) domain. A crucial property is that the functions ${Q}_{i}$, $i=1,\dots ,n$, in (9) can be used as Lyapunov functions that decrease along the RMNN equations in the voltage–current domain. This enables an association of an asymptotic system in the flux–charge domain—to which the Lyapunov approach can be effectively used to prove convergence—with an RMNN.
- From an the point of view of applications, an RMNN can be used to process signals and images in the flux–charge domain, i.e., the dynamics of the memristor fluxes can be used instead of using the dynamics of capacitor voltages, as happens for traditional memristor-less NNs operating in the voltage–current domain. A simple application to an image processing task will be illustrated in Section 7. We stress that in an RMNN, memristors are used in the analog computation, but they are also able to store the computational result, i.e., the asymptotic values of fluxes, in accordance with the principle of in-memory computing.

**Proof of**

**Theorem 1.**

**Property**

**7.**

**Proof.**

## 7. Numerical Simulations and Application

## 8. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Li, S.; Xu, L.D.; Zhao, S. The internet of things: A survey. Inf. Syst. Front.
**2015**, 17, 243–259. [Google Scholar] [CrossRef] - Waldrop, M.M. The chips are down for Moore’s law. Nat. News
**2016**, 530, 144. [Google Scholar] [CrossRef][Green Version] - Williams, R.S. What’s next? [The end of Moore’s law]. Comp. Sci. Eng.
**2017**, 19, 7–13. [Google Scholar] [CrossRef] - Krestinskaya, O.; James, A.P.; Chua, L.O. Neuromemristive circuits for edge computing: A review. IEEE Trans. Neural Netw. Learn. Syst.
**2019**, 31, 4–23. [Google Scholar] [CrossRef] [PubMed][Green Version] - Yang, J.J.; Williams, R.S. Memristive devices in computing system: Promises and challenges. ACM J. Emerg. Technol. Comput. Syst. (JETC)
**2013**, 9, 11. [Google Scholar] [CrossRef] - Zidan, M.A.; Strachan, J.P.; Lu, W.D. The future of electronics based on memristive systems. Nat. Electron.
**2018**, 1, 22. [Google Scholar] [CrossRef] - Sebastian, A.; Gallo, M.L.; Burr, G.W.; Kim, S.; BrightSky, M.; Eleftheriou, E. Tutorial: Brain-inspired computing using phase-change memory devices. J. Appl. Phys.
**2018**, 124, 111101. [Google Scholar] [CrossRef][Green Version] - Li, C.; Wang, Z.; Rao, M.; Belkin, D.; Song, W.; Jiang, H.; Yan, P.; Li, Y.; Lin, P.; Hu, M.; et al. Long short-term memory networks in memristor crossbar arrays. Nat. Mach. Intell.
**2019**, 1, 49. [Google Scholar] [CrossRef][Green Version] - Xia, Q.; Yang, J.J. Memristive crossbar arrays for brain-inspired computing. Nat. Mater.
**2019**, 18, 309–323. [Google Scholar] [CrossRef] - Ielmini, D.; Pedretti, G. Device and circuit architectures for in-memory computing. Adv. Intell. Syst.
**2020**, 2, 2000040. [Google Scholar] [CrossRef] - Sebastian, A.; Gallo, M.L.; Khaddam-Aljameh, R.; Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol.
**2020**, 15, 529–544. [Google Scholar] [CrossRef] [PubMed] - Chua, L.O.; Kang, S.M. Memristive devices and systems. Proc. IEEE
**1976**, 64, 209–223. [Google Scholar] [CrossRef] - Chua, L. Everything you wish to know about memristors but are afraid to ask. Radioengineering
**2015**, 24, 319–368. [Google Scholar] [CrossRef] - Corinto, F.; Forti, M.; Chua, L.O. Nonlinear Circuits and Systems with Memristors; Springer Nature Switzerland AG: Cham, Switzerland, 2021. [Google Scholar]
- Chua, L.O. Memristor-The missing circuit element. IEEE Trans. Circuit Theory
**1971**, 18, 507–519. [Google Scholar] [CrossRef] - Mazumder, P.; Kang, S.M.; Waser, R. Special issue on memristors: Devices, models, and applications. Proc. IEEE
**2012**, 100, 1911–1919. [Google Scholar] [CrossRef] - Ascoli, A.; Corinto, F.; Senger, V.; Tetzlaff, R. Memristor model comparison. IEEE Circuits Syst. Mag.
**2013**, 13, 89–105. [Google Scholar] [CrossRef] - Kvatinsky, S.; Friedman, E.G.; Kolodny, A.; Weiser, U.C. TEAM: Threshold adaptive memristor model. IEEE Trans. Circuits Syst. I Reg. Pap.
**2013**, 60, 211–221. [Google Scholar] [CrossRef] - Hajri, B.; Aziza, H.; Mansour, M.M.; Chehab, A. RRAM device models: A comparative analysis with experimental validation. IEEE Access
**2019**, 7, 168963–168980. [Google Scholar] [CrossRef] - Cohen, M.A.; Grossberg, S. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Syst. Man Cybern.
**1983**, 13, 815–825. [Google Scholar] [CrossRef] - Hopfield, J.J. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. USA
**1984**, 81, 3088–3092. [Google Scholar] [CrossRef] - Hirsch, M. Convergent activation dynamics in continuous time networks. Neural Netw.
**1989**, 2, 331–349. [Google Scholar] [CrossRef][Green Version] - Chua, L.O.; Yang, L. Cellular neural networks: Theory. IEEE Trans. Circuits Syst.
**1988**, 35, 1257–1272. [Google Scholar] [CrossRef] - Zurada, J. Introduction to Artificial Neural Systems; West Publishing Co.: Eagan, MN, USA, 1992. [Google Scholar]
- Haykin, S. Neural Networks: A Comprehensive Foundation; Prentice-Hall: Hoboken, NJ, USA, 1999. [Google Scholar]
- Liu, P.; Wang, J.; Zeng, Z. An overview of the stability analysis of recurrent neural networks with multiple equilibria. IEEE Trans. Neural Netw. Learn. Syst.
**2021**. [Google Scholar] [CrossRef] [PubMed] - Michel, A.N.; Farrell, J.A.; Porod, W. Qualitative analysis of neural networks. IEEE Trans. Circuits Syst.
**1989**, 36, 229–243. [Google Scholar] [CrossRef] - Yang, S.; Liu, Q.; Wang, J. A collaborative neurodynamic approach to multiple-objective distributed optimization. IEEE Trans. Neural Netw. Learn. Syst.
**2018**, 29, 981–992. [Google Scholar] [CrossRef] [PubMed] - Arik, S. New criteria for stability of neutral-type neural networks with multiple time delays. IEEE Trans. Neural Netw. Learn. Syst.
**2020**, 31, 1504–1513. [Google Scholar] [CrossRef] - Di Marco, M.; Forti, M.; Grazzini, M.; Pancioni, L. Limit set dichotomy and multistability for a class of cooperative neural networks with delays. IEEE Trans. Neural Netw. Learn. Syst.
**2012**, 23, 1473–1485. [Google Scholar] [CrossRef] - Forti, M.; Tesi, A. Absolute stability of analytic neural networks: An approach based on finite trajectory length. IEEE Trans. Circuits Syst. I
**2004**, 51, 2460–2469. [Google Scholar] [CrossRef] - Di Marco, M.; Forti, M.; Pancioni, L. Complete stability of feedback CNNs with dynamic memristors and second-order cells. Int. J. Circuit Theory Appl.
**2016**, 44, 1959–1981. [Google Scholar] [CrossRef] - Di Marco, M.; Forti, M.; Pancioni, L. Convergence and multistability of nonsymmetric cellular neural networks with memristors. IEEE Trans. Cybern.
**2017**, 47, 2970–2983. [Google Scholar] [CrossRef] - Di Marco, M.; Forti, M.; Pancioni, L. Memristor standard cellular neural networks computing in the flux–charge domain. Neural Netw.
**2017**, 93, 152–164. [Google Scholar] [CrossRef] - Deng, K.; Zhu, S.; Bao, G.; Fu, J.; Zeng, Z. Multistability of dynamic memristor delayed cellular neural networks with application to associative memories. IEEE Trans. Neural Netw. Learn. Syst.
**2021**. [Google Scholar] [CrossRef] [PubMed] - Deng, K.; Zhu, S.; Dai, W.; Yang, C.; Wen, S. New criteria on stability of dynamic memristor delayed cellular neural networks. IEEE Trans. Cybern.
**2022**, 52, 5367–5379. [Google Scholar] [CrossRef] [PubMed] - Di Marco, M.; Forti, M.; Moretti, R.; Pancioni, L.; Innocenti, G.; Tesi, A. Convergence of a class of delayed neural networks with real memristor devices. Mathematics
**2022**, 10, 2439. [Google Scholar] [CrossRef] - Tetzlaff, R.; Ascoli, A.; Messaris, I.; Chua, L.O. Theoretical foundations of memristor cellular nonlinear networks: Memcomputing with bistable-like memristors. IEEE Trans. Circuits Syst. I Reg. Pap.
**2020**, 67, 502–515. [Google Scholar] [CrossRef] - Corinto, F.; Gilli, M.; Forti, M. Flux-charge description of circuits with non-volatile switching memristor devices. IEEE Trans. Circuits Syst. II Expr. Briefs
**2018**, 65, 642–646. [Google Scholar] [CrossRef] - Yang, J.J.; Pickett, M.D.; Li, X.; Ohlberg, D.A.A.; Stewart, D.R.; Williams, R.S. Memristive switching mechanism for metal/oxide/metal nanodevices. Nat. Nanotech.
**2008**, 3, 429. [Google Scholar] [CrossRef] - Hu, S.G.; Wu, S.Y.; Jia, W.W.; Yu, Q.; Deng, L.J.; Fu, Y.Q.; Liu, Y.; Chen, T.P. Review of nanostructured resistive switching memristor and its applications. Nanosci. Nanotechnol. Lett.
**2014**, 6, 729–757. [Google Scholar] [CrossRef] - Khalid, M. Review on various memristor models, characteristics, potential applications, and future works. Trans. Electr. Electron. Mater.
**2019**, 20, 289–298. [Google Scholar] [CrossRef] - Di Marco, M.; Forti, M.; Pancioni, L. New conditions for global asymptotic stability of memristor neural networks. IEEE Trans. Neural Netw. Learn. Syst.
**2018**, 29, 1822–1834. [Google Scholar] [CrossRef] - Corinto, F.; Forti, M. Memristor circuits: Flux–charge analysis method. IEEE Trans. Circuits Syst. I Reg. Pap.
**2016**, 63, 1997–2009. [Google Scholar] [CrossRef][Green Version] - Chua, L.O.; Yang, L. Cellular neural networks: Applications. IEEE Trans. Circuits Syst.
**1988**, 35, 1273–1290. [Google Scholar] [CrossRef]

**Figure 1.**Memristor models: Ideal flux-controlled memristor (

**left**) and real extended memristor given by the parallel connection of an ideal memristor and a nonlinear voltage-controlled resistor (

**right**).

**Figure 2.**Characteristics of ${\widehat{i}}_{\mathrm{R}}$ for a nonlinear voltage-controlled resistor. (

**a**) PWL diode satisfying Assumption 1. (

**b**) Exponential characteristic of a Shockley diode satisfying Assumption 2; the drawing also shows the tangent to ${\widehat{i}}_{\mathrm{R}}$ at ${v}_{\mathrm{R}}=0$ (dashed–dotted) and the function ${\widehat{i}}_{\mathrm{R}}^{+}$ (light gray). (

**c**) Characteristic with a negative slope at ${v}_{\mathrm{R}}=0$ satisfying Assumption 2 and (

**d**) non-monotone characteristic satisfying Assumption 2.

**Figure 3.**Circuit schematic for the i-th cell of an IMNN. The cell is obtained by replacing the linear resistor in a CNN cell with a flux-controlled ideal memristor. The interconnections are obtained via linear conductances ${g}_{ij}$. The VCCSs represent the currents ${g}_{ij}{v}_{j}$, $j=1,2,\dots ,n$, which are injected into the cell due to the interconnections with the other cells.

**Figure 4.**Circuit schematic for the i-th cell of an RMNN. The cell is obtained by replacing the ideal flux-controlled memristor in the cell of an IMNN (cf. Figure 3) with a real extended memristor given by the parallel connection of an ideal flux-controlled memristor and a nonlinear voltage-controlled resistor.

**Figure 5.**Numerical simulation with MATLAB of a memristor CNN for horizontal line detection. (

**a**) Initial image at $t=0$; (

**b**) snapshot of the transient at $t=0.1$; (

**c**) snapshot at $t=0.15$; (

**d**) final image at $t=1$.

**Figure 6.**Numerical simulation with MATLAB of a memristor CNN for horizontal line detection. (

**a**) Time evolution of fluxes ${\psi}_{i}(\xb7;{\phi}_{0},0)$ for the 13th row of the CNN; (

**b**) evolution of nonlinear resistor charges ${\mathcal{Q}}_{\mathrm{R},i}(\xb7;{\phi}_{0},0)$; (

**c**) evolution of fluxes ${\phi}_{i}(\xb7;{v}_{0},{\phi}_{0})$; (

**d**) evolution of capacitor voltages ${v}_{i}(\xb7;{v}_{0},{\phi}_{0})$.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Di Marco, M.; Forti, M.; Moretti, R.; Pancioni, L.; Tesi, A. Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics. *Mathematics* **2022**, *10*, 4024.
https://doi.org/10.3390/math10214024

**AMA Style**

Di Marco M, Forti M, Moretti R, Pancioni L, Tesi A. Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics. *Mathematics*. 2022; 10(21):4024.
https://doi.org/10.3390/math10214024

**Chicago/Turabian Style**

Di Marco, Mauro, Mauro Forti, Riccardo Moretti, Luca Pancioni, and Alberto Tesi. 2022. "Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics" *Mathematics* 10, no. 21: 4024.
https://doi.org/10.3390/math10214024