- This event has passed.
Benchmarking Big Data and Artificial Intelligence – Evaluation schemes for Big data and AI Performance of high Business impact
November 4, 2020 @ 10:00 am - 1:30 pm
For many years companies have been running technical benchmarks to compare the performance of different technologies and systems. This benchmarking process provides very valuable information but in many cases, it is not correlated with the impact of the solutions at business level. This was precisely the starting point of the DataBench project, funded by the EC under the Big Data Value PPP. After three years of intensive work DataBench has created an evaluation framework that is already openly available for the community. The session will introduce the different elements of the framework, including the toolbox, the way it can be used and some concrete examples run with companies and projects that work heavily with data.
- The current landscape of Big Data and AI benchmarks
- The DataBench framework, which includes a complete set of metrics for the assessment of Big Data technologies
- The DataBench toolbox, a web-based tool that provides a unique environment to search, select and deploy big data benchmarking tools, giving the possibility to generate unified technical metrics and derive business KPIs
- A comprehensive set of use cases that we have run with companies in different industrial domains and projects of the Big Data PPP to illustrate the way you can get value out of using DataBench
- Pipelines and blueprints
The DataBench toolbox follows a platform model and provides a single place where benchmarking communities and users of such benchmarks can meet. It goes beyond any work done so far by translating major technical KPIs into business KPIs, allowing us to establish relationships between technical and business decisions. As such, the session targets a multiplicity of stakeholders: benchmark providers, companies interested in benchmarking big data, projects and members of the big data and AI communities, decision-makers of companies that are thinking about their data-driven transformation and policy makers looking for evidence-based decision-support tools.
On our side, we will bring a comprehensive set of speakers falling precisely under all those categories.
The workshop will be structured around two parts (each of them can be attended independently; like this we want to facilitate that those participants that are not available for the entire workshop, select the most interesting part for them). Attending the complete workshop is in any case recommended to take maximum advantage of the contents.
10:00-10:05 Intro. Objectives of the session (Nuria de Lama (Atos)
10:05-10:15 DataBench General Overview (Richard Stevens (IDC, DataBench coordinator)
PART I. Big Data Benchmarking landscape and Big Data Pipelines
- 10:15-11:15 Session 1. The current landscape of Big Data benchmarks
10:15-10:25: DataBench Framework for Benchmarks, Arne J. Berre, SINTEF
10:25-10:40: Hobbit, Benchmarking Platforms and AI, Axel Ngonga, BDVA TF6 Benchmark Lead, University of Paderborn
10:40-10:55: BenchCouncil Big Data and AI Benchmarks, Wanling Gao, Chinese Academy of Sciences
10:55-11:10: MLPerf AI and ABench, Rekha Singhal, Senior Scientist and Head of the Computing Systems-Software Research area at TCS
11:10-11:15: Conclusion on Big Data and AI Benchmarks, Todor Ivanov, LeadConsult
- 11:15-12:15 Session 2. A Project perspective on Big Data and AI architectural pipelines and benchmarks
11:15-11:20: Introduction to Architectural pipelines, Arne J. Berre, SINTEF
11:20-11:30: I- BiDaaS – Leonidas Kallipolitis, AEGIS
11:30-11:40: TBFY – Brian Elvesæter, SINTEF
11:40-11:50: Track&KNow – Athanasios Koumparos, Vodafone
11:50-12:00: DataBio – Caj Södergård, VTT
12:00-12:10: DeepHealth – Jon Ander Gómez Adrián, Universitat Politecnica de Valencia
12:10-12:15: Conclusion on Pipelines and related benchmarks, Arne J. Berre, SINTEF
12:15-12:30 Short coffee break to relax and maybe grab a coffee
PART II. Big Data Business Framework and benchmarking tooling support
- 12:30-13:00 Session 3. The DataBench Business framework: a compelling offering to measure the impact of Big Data Technologies
12.30-12.40: The DataBench business framework, by Gabriella Cattaneo, Erica Spinoni and Chiara Francalanci
12: 40-12.50: The Whirlpool use case, Pierluigi Petrali (Whirlpool)
12.50-13.00: The Fill use case, Harald Sehrschön (FILL)
- 13:00-13:30 Session 4. A practical journey on how to use the DataBench Toolbox
13:00-13:15 Demo of the DataBench Toolbox (Tomás Pariente, Atos)
13:15-13:25 AI Observatory (Marko Grobelnik, JSI)
13:25-13:30 Fostering adoption of DataBench results. Needs from the point of view of Digital Innovation Hubs (Daniel Sáez, ITI, EuHubs4Data coordinator)
- 13:30 Concluding Remarks and closing of the session