Call for Collaboration: Data for Machine Learning

Alberto Troccoli

November 27, 2019 at 1:38 pm #10529

Please refer to Item 4 of the Data SIG meeting (18/11/2019) minutes for context and the discussion prior to the opening of this forum topic.

Sue-Ellen Haupt reported on initiatives focusing on trying to bring together to form useful datasets that can be used for testing machine learning methods and to do inter-comparisons. Sharing her thoughts, particularly from a workshop held in Oxford in September, Sue highlighted interest in a platform called “Pangeo” (https://pangeo.io/index.html), something NCAR and the UK Met Office is working with. It could be an ‘ideal platform’ for data sharing for the purposes of machine learning – a place to host some renewable energy datasets; to bring renewable energy as one of the topics that we could be testing for on this platform.

In Oxford, Sue was part of a group on post-processing of weather and climate information, which is an area where ‘we’re furthest along on using AI in climate and weather’. WEMC members in renewable energy are very familiar with the type of post-processing, specifically for things like renewable energy forecasting: using AI to build post-processors to predict either hub height, wind speed or irradiance that hits a PV panel – things that are needed to be able to integrate renewable energy into the grid better.

There will be a call going to be put out a request for quality data sets and then for an inter-comparison study of different techniques. It may be found that different techniques will be dependent on the location, on the type of data, on the quality of data, the amount of data, etc, and Sue would love to involve people from the Data SIG as many are working on similar datasets, contributing those data sets and then testing them for our machine learning methods.

Questions for discussion on this forum topic:

1. Do you have datasets on different scales?
2. Would you be interested in either contributing those datasets, or testing your own machine learning methods on data sets that are conscripted?
3. What are your thoughts for a collaborate data approach to aid machine learning? (e.g. the potential benefits/barriers?)

 

Thank you to Gerald van der Grijn, Jan Dutton, Alberto Troccoli, Laurent Dubus and Jose-Luis Casado for their thoughts during the meeting (minuted).