Chasing Storms through Terabytes of Data
Toolkit lets scientists detect extreme weather in climate simulations far faster than before.
The Science
Scientists wanted to detect and track tropical cyclones and other extreme weather events in terabyte-sized computer simulations. In response, a team built TECA. It stands for the Toolkit for Extreme Climate Analysis.
The Impact
Tropical storms are one of the most damaging climate events in terms of monetary destruction and loss of human life. By using TECA on supercomputers, scientists can analyze computational models and changes in tropical storm system frequency in a matter of hours.
Summary
Climate simulations create massive amounts of data (“big data”), requiring sophisticated pattern recognition. Simulations that today produce tens of gigabytes of data will soon be modified to produce tens of terabytes per time step. This dramatic increase in data size would present a significant challenge to the climate community. When coupled with increased complexity of the models, timely analysis of the data could become insurmountable. A team from Lawrence Berkeley and Argonne National Laboratories used TECA on the National Energy Research Scientific Computing Center’s (NERSC) Hopper and Argonne Leadership Computing Facility’s Mira systems to analyze 56 terabytes of climate data on nearly 1,000,000 supercomputer cores. The data was from the fifth phase of the Coupled Model Intercomparison Project. They identified three classes of storms: tropical cyclones, atmospheric rivers, and extra-tropical cyclones. They preprocessed the data on the Hopper system at NERSC. Preprocessing took about two weeks and resulted in a final 15-terabyte dataset. At the 16th International Conference on Computer Analysis of Images and Patterns, held in September 2015, TECA was honored with the Juelich Supercomputing Center prize. The prize honors the best application of high-performance computing technology in solving a pattern recognition problem.
Contact
Prabhat
Lawrence Berkeley National Laboratory
[email protected]
Funding
This material is based on work supported by the Department of Energy, Office of Science, Office of Biological and Environmental Research, Climate and Environmental Sciences Division, Regional & Global Climate Modeling Program. The computational resources of the National Energy Research Scientific Computing Center (NERSC, Lawrence Berkeley National Laboratory) and the Argonne Leadership Computing Facility (ALCF, Argonne National Laboratory), both of which are Office of Science scientific user facilities, were used.
Publications
Prabhat, S. Byna, V. Vishwanath, E. Dart, M. Wehner, and W.D. Collins, “TECA: Petascale Pattern Recognition for Climate Science.” In: G. Azzopardi and N. Petkov (eds), Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, 9257, 426 (2015). [DOI: 10.1007/978-3-319-23117-4_37]
Related Links
National Energy Research Scientific Computing Center news article: Berkeley Lab Climate Software Honored for Pattern Recognition Advances
Highlight Categories
Performer: DOE Laboratory , SC User Facilities , ASCR User Facilities , NERSC , ALCF
