PhD defense Fatma Kiraz: Resistive Neural Networks Learning by Time-Continuous Analog Computation
Télécom Paris, 19 place Marguerite Perey F-91120 Palaiseau [getting there], amphi 5 and in videoconferencing
Jury
- Dominique DALLET, Full Professor, Bordeaux INP, Laboratoire IMS (Reviewer)
- Haralampos STRATIGOPOULOS, Research Director, LIP6, CNRS, Sorbonne Université (Reviewer)
- Damien QUERLIOZ, Research Director, CNRS, Université Paris-Saclay (Examiner)
- Pietro Maris FERREIRA, Full Professor, Université Savoie Mont Blanc (Examiner)
- Dang-Kién Germain PHAM, Associate Professor, Télécom Paris (Thesis co-supervisor)
- Patricia DESGREYS, Full Professor, Télécom Paris (Thesis supervisor)
Abstract
Equilibrium Propagation (EqProp) presents a promising approach for training analog neural networks by estimating error gradients, offering a hardware-friendly, brain-inspired algorithm. Grounded in the mathematical framework of energy-based models (EBMs), EqProp defines the relationship between the states of a system, its inputs, and a set of parameters through an energy function. Nonlinear resistive networks can be effectively trained using EqProp as being an EBM. This thesis focuses on implementing the EqProp algorithm on nonlinear resistive networks with enhanced training efficiency and accuracy.
… that the EqProp algorithm is sensitive to learning parameters and randomized starting conductance values. We demonstrate the negligence of certain analog components—like diodes—in energy function and ignore the inherent interdependencies in analog circuits that might be causing the chaotic behavior. We propose a modified approach grounded in Kirchhoff’s laws to account for the interdependencies in analog circuits. Based on this, we introduce a different energy function based on conventional electrical power by accounting for all relevant components and their effects on the optimization process. Additionally, this thesis presents a top-down analysis beginning with the conceptual framework of machine learning models. This approach guides the design of our analog circuit architecture and ensures a natural transition from software to hardware while addressing the limitation of positive conductance. Extensive PySpice simulations demonstrate the superior performance of our method, achieving significant error rate reductions and robust, stable learning across various initial resistance configurations. This work underscores the potential of using a more accurate energy function to enhance the efficiency and robustness of EqProp in training nonlinear resistive networks.