A Common and Sustainable Big Data Infrastructure in Support of Weather Prediction Research and Education in Universities

The goal of the Big Weather Web is to make big data infrastructure affordable and adequate for university members of the Numerical Weather Prediction community by combining the application of three recent technologies: virtualization, cloud computing and storage, and big data management*.

Virtualization allows push-button deployment and maintenance of complex systems, smart storage provides automatic, community-wide data availability guarantees, and big data management allows for easy curation of data and its products. The combination of these three technologies allows communities to create a standard community-specific computational environment and efficiently refine it with minimal repetition of work, introducing a high degree of reproducibility in research and education. This reproducibility accelerates learning and amplifies everyone’s contribution.

The Big Weather Web will be developed in the context of numerical weather prediction with the expectation that the resulting infrastructure can be easily adapted to other data-intensive scientific communities.

(*) An earlier version stated: virtualization, federated smart storage, and big data management.