Session: 07-17-01: Machine Learning and Artificial Intelligence in Dynamics, Vibrations and Control

Paper Number: 144621

144621 - Symbolic Regression via Neural Networks 

Identifying governing equations for a dynamical system is a topic of critical interest across an array of disciplines, from mathematics to engineering to biology. Machine learning - specifically deep learning - techniques have shown their capabilities in approximating dynamics from data, but a shortcoming of traditional deep learning is that there is little insight into the underlying mapping beyond its numerical output for a given input. This limits their utility in analysis beyond simple prediction. Simultaneously, a number of strategies exist which identify models based on a fixed dictionary of basis functions, but most either require some intuition or insight about the system, or are susceptible to overfitting or a lack of parsimony. Here, we present a novel approach that combines the flexibility and accuracy of deep learning approaches with the utility of symbolic solutions: a deep neural network that generates a symbolic expression for the governing equations. We first
describe the architecture for our model and then show the accuracy of our algorithm across a range of classical dynamical systems.

The models are built up from primitive operations, such as addition, multiplication, exponentiation, and other user-defined functions of state variables that commonly occur in dynamical systems. Drawing inspiration from Deep Neural Networks (DNNs), we allow the breadth of the network to determine the number of allowable coefficients in a certain set of functions and the depth to determine compositions among them.  Our approach allows the generation of interpretable functions of state variables, including linear combinations, polynomials, and non-polynomial functions, such as fractional and negative powers, logistic functions, and expressions that might arise in chemical kinetics or a conductance-based model for neural activity. The complexity of these generated functions that appear in the model is determined directly by the depth and width of the neural network rather than having to pre-specify them exactly. Our neural network architecture is novel in how the weights of the network determine the form of the terms in the generated dictionary, which allows a wider variety of model terms than other recent system identification approaches that use neural networks. Moreover, the number of terms in the generated dictionary remains small enough to handle computationally, but large enough to capture a rich set of possibilities. Available algorithms for optimizing DNNs are used to optimize the coefficients of state variables in these expressions. To obtain parsimonious models, we use a combination of sparsity-inducing regularization and the Akaike Information Criterion, which selects between candidate models, obtained through perturbations of model parameters within user-specified tolerances, to balance model simplicity and accuracy. We call this SymANNTEx (pronounced as “semantics”), for Symbolic, Artificial Neural Network-Trained Expressions.

Presenting Author: Jeff Moehlis University of California, Santa Barbara

Presenting Author Biography: Jeff Moehlis received a Ph.D. in Physics from UC Berkeley in 2000, and was a Postdoctoral Researcher in the Program in Applied and Computational Mathematics at Princeton University from 2000-2003. He joined the Department of Mechanical Engineering at UC Santa Barbara in 2003, and is currently Chair of this department. He was also recently the Chair of the Program in Dynamical Neuroscience at UC Santa Barbara. He has been a recipient of a Sloan Research Fellowship in Mathematics and a National Science Foundation CAREER Award, and was Program Director of the SIAM Activity Group in Dynamical Systems from 2008-2009. Jeff's current research includes applications of dynamical systems and control techniques to neuroscience, cardiac dynamics, and collective behavior. He has published over 100 journal / conference proceedings articles on these and other topics including shear flow turbulence, microelectromechanical systems, energy harvesting, and dynamical systems with symmetry.

Authors:

Nibodh Boddupalli University of California, Santa Barbara
Tim Matchen University of California, Santa Barbara
Jeff Moehlis University of California, Santa Barbara