In the era of Big Data, large and complex data sets from different sources need to be combined and processed in order to translate data into knowledge. Properly identifying the origin and nature of such data sets, together with strict tracking of the processing steps applied, is essential to establish trust in the generated products in both public research, where reproducibility of results is a must, and in the private sector, where the choice of an erroneous or unreliable product could lead to losses of millions of dollars. This practice, also known as data provenance, is and has been studied by the Research Data Alliance (RDA), especially by the dedicated Provenance Patterns Working Group. For Earth science applications in particular, with the advent of operational climate services, an urgent need has been identified to develop more robust data provenance practices. For example, in climate forecast verification the results are highly sensitive to the choice of climate data sets used, as well as the specific operations applied to these data sets. The user needs to be able to identify how a verification metric of a forecast was obtained to have confidence in such forecast product.
RDA-Europe and the Barcelona Supercomputing Center are organizing a 2-day meeting with the objective of bringing together interested parties from the public and private sectors to discuss the main challenges in data provenance. This meeting will cover approaches to data provenance at the European level in order to identify any relevant open issues, make all initiatives aware of similar ongoing efforts, find commonalities between initiatives, and define some guidelines to make these efforts complement each other in the best way possible. These issues will be considered from an Earth sciences perspective, but contributions from other research fields will be welcome to ensure the much-needed cross-breeding.
See agenda here.