DOE seeds first-principles study of AI and deep learning, part of $4.3 million national initiative

Kahn, Shelton, and Hooberman to characterize AI deep neural networks for scientific research

Today, the U.S. Department of Energy (DOE) announced $4.3 million in funding for 16 projects that will use artificial intelligence (AI) to spur advances in high-energy physics and use tools from high-energy physics to better understand how AI works. Among these, a team of physicists at the University of Illinois Urbana-Champaign will apply tools from theoretical high-energy physics to develop a first-principles understanding of the uncertainty inherent in analyses using deep neural networks.  

The 3-year awards support the DOE Office of Science initiative in artificial intelligence research aimed at using AI techniques to deliver scientific discoveries that would not otherwise be possible and broadening participation in high energy physics research.

Gina Rameika, DOE Associate Director of Science for High Energy Physics, comments in the DOE announcement, “AI and Machine Learning (ML) techniques in high energy physics are vitally important for advancing the field. These awards represent new opportunities for university researchers that will enable the next discoveries in high energy physics.”

Illinois Physics Professor Yonatan Kahn is primary investigator on the Illinois project, titled “Uncertainty Quantification from Neural Network Correlation Functions.” He and co-PIs Illinois Physics Professors Jessie Shelton and Ben Hooberman each brings a graduate student to the team—Hannah Day, Victoria Tiki, and Kai Zheng, respectively. The team also includes Kahn’s longtime collaborator Dan Roberts, principal researcher at Salesforce and a research affiliate at the Center for Theoretical Physics at the Massachusetts Institute of Technology.

Over the past decade, commercial use of artificial intelligence (AI) has become commonplace, thanks to the development of deep neural networks. AI and deep learning have enabled computers that can beat the most highly skilled humans at games such as Go and poker. It lets us give commands to or ask questions of Siri or Alexa and expect reasonably accurate responses. It enables banks to automate the processing of handwritten checks uploaded as photos from our smartphones. It supports the U.S. Postal Service’s automated sorting and routing, even of hand-addressed letters and parcels. And it enables facial recognition for security locks on devices, for tagging people on social media, and for reverse image searches in web browsers.

This advanced technology—which has coevolved with faster, more powerful computer processing capabilities and memory storage capacities—is able to teach itself pattern recognition at a level of abstraction more complex and nuanced than any computer programmer could code.

Kahn explains, “AI and deep learning allow machines to learn from data in a way that we don’t have to explicitly program—the neural network infers the rules it uses to identify and process information.”

Deep learning neural networks and deep learning algorithms mirror how the human brain processes and understands complex information. Such neural networks comprise interconnected artificial neurons in distinct processing layers that, at the highest processing levels, can achieve the same kind of abstraction of concepts that the human mind relies on to make sense of the world. Both language and visual processing entail successfully identifying variable patterns of a particular thing as different examples of the same abstract concept.

Just as AI and deep learning have transformed the way we interact with technology, they equally holds tremendous promise for advancing scientific research, especially in the big-data sciences like high-energy physics (HEP) and astrophysics. But before that potential can be fully realized, deep learning must be more precisely characterized. Unlike commercial enterprises, science is only as good as it is exact. But given our current theoretical understanding of AI, the uncertainty (or “percent accuracy”) of any given deep learning analysis must be estimated.

Kahn notes, “AI tools have demonstrated amazing capabilities, but we don’t really know how they work—they’re a fairly mysterious set of black boxes. By using tools from theoretical physics to open the black boxes a little, we hope to be able to improve our understanding of AI, in particular by quantifying the range of possible answers a deep neural network provides for a given question.”

Kahn and his team will use calculational tools and intuition from field theory to quantify neural network uncertainties, without having to train thousands of networks.

Kahn explains, “Once trained, deep learning neural networks have so many parameters, we couldn’t ever adequately describe them. In this research, we will use a set of ideas from physics that boils down to this—the more parameters there are, the less complicated the system. This is because, in statistical mechanics, the relative size of fluctuations gets smaller the more information we have.”

The team will do some of the work for this project on standard computers, using scaled down models. Additional work will be done at the National Center for Supercomputing Applications (NCSA). Kahn expressed gratitude for a letter of support for his project, penned by NCSA’s Volodomyr Kindratenko, director of the Center for Artificial Intelligence Innovation.

Kahn says the team plans to apply its uncertainty quantification tools to tasks such as particle identification at high-energy collider facilities and characterization of dark matter halo properties in astrometric data collected by the Gaia space observatory of the European Space Agency.

The DOE’s announcement for seed awards to universities follows last month’s announcement of team awards to national laboratories, part of the same funding initiative: Artificial Intelligence Research for High Energy Physics DE-FOA-0002705. The winning projects were selected by competitive peer review.

Total funding is $4.3 million for the 16 projects, with $1.3 million in the first budget period. The list of projects and more information can be found here.