New “AI scientists” combine theory and data to discover scientific equations

The goal of scientists is to discover meaningful formulas that accurately describe experimental data. Mathematical models of natural phenomena can be created manually based on domain knowledge, or they can be created automatically from large data sets using machine learning algorithms. The academic community has studied the problem of merging related prior knowledge and related function models, and believes that finding a model that is consistent with prior knowledge of general logical axioms is an unsolved problem.

Researchers from the IBM research team and Samsung AI team developed a method "AI-Descartes" that combines logical reasoning with symbolic regression to extract data from axiomatic knowledge and experimental data. in principle derivation of models of natural phenomena.

The research is titled "Combining data and theory for derivable scientific discovery with AI-Descartes" and was published on April 12, 2023 in "Nature Communications》.

Artificial neural networks (NN) and statistical regression are often used to automatically discover patterns and relationships in data. NN returns a "black box" model, where the underlying functions are typically used only for prediction. In standard regression, the functional form is predetermined, so model discovery amounts to parameter fitting. In symbolic regression (SR), the functional form is not predetermined but consists of operators from a given list (e.g., , -, ×, and ÷) and is calculated from the data.

SR models are generally more "interpretable" than NN models and require less data. Therefore, to discover natural laws symbolically from experimental data, SR may be more effective than NN or fixed-form regression; the integration of NN and SR has been the subject of recent research in neurosymbolic AI. A major challenge in SR is identifying scientifically meaningful models from the many models that fit the data. Scientists define a meaningful function as one that balances accuracy and complexity. However, many such expressions exist for a given data set, and not all of them are consistent with known background theory.

An alternative approach is to start with a known background theory, but there are currently no practical inference tools that can generate theorems consistent with experimental data from a known set of axioms. Automatic Theorem Provers (ATP) are the most widely used reasoning tools that can prove conjectures for a given logical theory. Computational complexity is a major challenge for ATP; for some types of logic, proving conjectures is undecidable.

Additionally, deriving models from logical theories using formal reasoning tools is especially difficult when arithmetic and calculus operators are involved. Machine learning techniques have been used to improve the performance of ATP, for example, by using reinforcement learning to guide the search process.

Derivable models must not only be empirically accurate, but they should also be predictive and insightful.

Researchers from the IBM Research Team and Samsung AI Team attempted to obtain such a model by combining a novel mathematical optimization-based SR method with an inference system. This resulted in an end-to-end discovery system "AI-Descartes" that extracts formulas from data via SR and then provides a proof of the formula's derivability from a set of axioms, or provides a proof of inconsistency. When a model is provably not derivable, the researchers propose new measures that indicate how close the formula is to a derivable formula, and use their inference system to calculate the values ​​of these measures.

Illustration: System overview. (Source: paper)

In early work combining machine learning with inference, scientists used logic-based descriptions to constrain the output of GAN neural architectures that generated images. There are also teams that combine machine learning tools and inference engines to search for functional forms that satisfy pre-specified constraints. This is to augment the initial data set with new points, thus improving the efficiency of the learning method and the accuracy of the final model. Some teams also leverage prior knowledge to create additional data points. However, these studies only considered constraints on the functional form to be learned and did not include general background theoretical axioms (logical constraints describing other laws and unmeasured variables involved in the phenomenon).

Cristina Cornelio, lead author of the paper and a research scientist at Samsung AI, said AI-Descartes offers some advantages over other systems, but its most distinguishing feature is that it logical reasoning ability. If there are multiple candidate equations that fit the data well, the system identifies which equation best fits the background scientific theory. The ability to reason also sets the system apart from "generative AI" programs like ChatGPT, which have limited logic capabilities in large language models and sometimes mess with basic math.

"In our work, we are combining first-principles methods with the more common data-driven methods of the machine learning era, which have been used by scientists for centuries. "This combination allows us to leverage both approaches and create more accurate and meaningful models for a wide range of applications."

The name AI-Descartes is a tribute to the 17th-century mathematician and philosopher René Descartes, who believed that the natural world could be described by a few basic physical laws and that logical inferences played a key role in scientific discoveries. .

Illustration: Explanation of the scientific method for system implementation. (Source: Paper)

Researchers from this team have demonstrated that combining logical reasoning with symbolic regression is of great value in obtaining meaningful symbolic models of physical phenomena. ; because they are consistent with background theory and generalize well to domains significantly larger than experimental data. The combination of regression and inference produces better models than either SR or logical inference alone.

Improvement or replacement of individual system components and the introduction of new modules, such as abductive inference or experimental design will expand the functionality of the entire system. Deeper integration of inference and regression can help synthesize data-driven and first-principles-based models and lead to a revolution in the scientific discovery process. Discovering models that are consistent with prior knowledge will accelerate scientific discovery and transcend existing discovery paradigms.

The team used models to deduce Kepler's third law of planetary motion, Einstein's relativistic time dilation law, and Langmuir's adsorption theory; the research shows that when logical reasoning is used to When distinguishing candidate formulas with similar errors on the data, the model can discover dominant patterns from a small number of data points.

Illustration: Visualization of related sets and their distances. (Source: paper)

# "In this work, we need human experts to write down in a formal, computer-readable way what the axioms of the background theory are, and if If humans miss any of them or get any of them wrong, the system won't work," said Tyler Josephson, assistant professor of chemistry, biochemistry and environmental engineering at UMBC. "In the future, we also hope to automate this part of the job so we can Explore more fields of science and engineering."

Ultimately, the team hopes their AI-Descartes can inspire a productive new scientific approach just like real scientists. "One of the most exciting aspects of our work is the potential for significant advances in scientific research," Cornelio said.

Paper link: https://www.nature.com/articles/s41467-023-37236-y

Related reports: https://techxplore.com/news/2023-04-ai-scientist-combines-theory-scientific.html

The above is the detailed content of New “AI scientists” combine theory and data to discover scientific equations. For more information, please follow other related articles on the PHP Chinese website!