Hybrid SORS/WiCS: How scientific workflows help automate science and what we can do better?

Download here the slides of the presentation.

Abstract: Scientific workflows are now a common tool used by domain scientists in several disciplines. They are appealing because they enable users to think at high level of abstraction, composing complex applications from individual application components. Workflow management systems (WMSs), such as Pegasus (http://pegasus.isi.edu) automate the process of executing these workflows on modern cyberinfrastructure. They take these high-level, resource-independent descriptions and map them onto the available heterogeneous resources: campus clusters, high-performance computing resources, high-throughput resources, clouds, and the edge. WMSs can select the appropriate resources based on their architecture, availability of key software, performance, reliability, availability of cycles, storage space, among others. With the help of compiler-inspired algorithms, they can determine what data to save during execution, and which are no longer needed. Similarly, to compiler solutions, they can generate an executable workflow that is tailored to the target execution environment, taking into account reliability, scalability, and performance. WMS use workflow execution engines to run the executable workflows on the target resources providing scalability and reliability. This talk will describe the key concepts used in the Pegasus WMS to help automate the execution of workflows in distributed and heterogeneous environments. It will explore potential use of artificial intelligence and machine learning approaches to enhance automation. The talk will also help identify challenges that exist in adopting novel approaches for science at the technological and social levels.

Short bio: Ewa Deelman received her PhD in Computer Science from the Rensselaer Polytechnic Institute in 1998. Following a postdoc at the UCLA Computer Science Department she joined the University of Southern California’s Information Sciences Institute (ISI) in 2000, where she is serving as a Research Director and is leading the Science Automation Technologies Center. She is also a Research Professor at the USC Computer Science Department and an AAAS, IEEE, and USC/ISI Fellow. The USC/ISI Science Automation Technologies Center explores the interplay between automation and the management of scientific workflows that include resource provisioning and data management. Dr. Deelman pioneered workflow planning for computations executing in distributed environments. Her group has led the design and development of the Pegasus Workflow Management software and conducts research in job scheduling and resource provisioning in distributed systems, workflow performance modeling, provenance capture, and the use of cloud platforms for science. Dr. Deelman is also the Principal Investigator and Director of the NSF-funded Cyberinfrastructure Center of Excellence that serves the NSF Major Facilities.
In 2006 Dr. Deelman founded the annual Workshop on Workflows in Support of Large-Scale Science (WORKS), which is held annually in conjunction with the SC conference. In 2015, Dr. Deelman received the HPDC Achievement Award for her contributions to the area of scientific workflows and in 2022 she received the Euro-Par Achievement Award for her outstanding contributions to parallel computing.