Finding the Milky Way's next exploding star using Agile software

Finding the Milky Way's next exploding star with Agile software

Stellar explosions – or supernovae – are commonplace in the Universe, and astronomers have been fortunate to discover thousands of them to date. However, in our own galaxy, supernovae are expected to occur only one to three times per century, and none have happened for over 400 years. The nearest well-studied supernova was in a nearby galaxy; its proximity to Earth allowed us to examine the explosion using both light and neutrinos for the first time. If such a supernova occurred in our own galaxy, it would be detectable not only via light and neutrinos, but also gravitational waves.

Studying an astronomical event – such as a supernova – through multiple signals is called "Multi-Messenger Astronomy" (MMA). MMA provides a new lens on cosmic events and brings a multidisciplinary approach to physics: fully understanding a supernova requires electromagnetic astronomy, neutrino physics, and gravitational-wave astrophysics. However, MMA relies on rapid communication between astrophysics experiments worldwide, which is possible only with extremely reliable, low-latency software linking observatories and detectors to quickly alert scientists of MMA events. To address this need, the Scalable Cyberinfrastructure for Multi-Messenger Astrophysics (SCiMMA) initiative was created in 2019. SCiMMA is an NSF project that comprises over 50 astrophysicists and computational scientists from over 10 institutions, including the University of Illinois Urbana-Champaign and the National Center for Supercomputing Applications.

I had been working with SCiMMA as a part-time software developer when we were contacted by the SuperNova Early Warning System (SNEWS), a project to discover the next galactic supernova via a network of ~20 neutrino detectors. The first version of the SNEWS MMA system had been developed by scientists in the early 2000s, but the growing number of neutrino experiments meant it needed to be modernized. SCiMMA’s systems were designed with modern software engineering best-practices, but they would need to be adapted to meet SNEWS’ specialized scientific needs. After discussion with principal investigators of each team, I began leading a 10-week remote collaboration between SCiMMA and SNEWS to develop a prototype of the new SNEWS 2.0 system, fully described in a paper published in Wiley Software: Practice and Experiences.

I orchestrated the collaboration using a method called Agile, a software development and project management framework that emphasizes short programming tasks to rapidly develop software from the bottom-up. The iterative strategy of small tasks contrasts with other methods wherein development is driven by goals that are planned up-front, or worse, never defined at all. This approach has made Agile popular in corporate and start-up environments, but it is rarely utilized in the astrophysics community.

Nonetheless, we found that the iterative approach of Agile was well-suited for collaboratively developing scientific software. Each week, we met virtually to plan and assign tasks to each person; each day, we would send a one-sentence summary of our work to the group. Our development efforts centered around a virtual "project board" that contained our main goals and the tasks we were planning, working on, or had completed. The project board helped us visualize the software requirements and our progress over time, which allowed us to identify any issues that lingered between weeks. This process ensured our weekly efforts matched our overall goals while also engaging both teams in the development process.

The expertise of the SCiMMA developers ensured that the software was well-designed, and the scientific knowledge of the SNEWS team guided the requirements for the software itself. The Agile framework facilitated the communication and coordination required for collaborative, remote software development. This allowed us to complete the prototype in a sustainable manner, letting the SNEWS team continue development on their own. In addition to supporting SNEWS' scientific effort, our work serves as a successful example of Agile software development in a discipline that has not traditionally used Agile. It also inspired SNEWS to adopt an Agile approach during their month-long summer workshop after the collaboration, where the prototype we developed was deployed across their detector network. While we still haven't seen a Milky Way supernova yet, SNEWS is ready for whenever it happens.