Big Data Benchmarking Communities
HOBBIT aims at abolishing the barriers in the adoption and deployment of Big Linked Data by European companies, by means of open benchmarking reports that allow them to assess the fitness of existing solutions for their purposes. These benchmarks are based on data that reflects reality and measures industry-relevant KPIs with comparable results using standardized hardware.
Task force 6 of BDVA focuses on technical aspects and standards for big data. Its main objective is to identify applications in which big data technologies can create the largest impact in Europe. SG7 (Special Group 7) will focus on big data benchmarking and will merge the efforts of several data benchmarking projects under one umbrella within which HOBBIT will take the lead of linked data benchmarking.
MLPerf’s mission is to build fair and useful benchmarks for measuring training and inference performance of ML hardware, software, and services. We believe that a widely accepted benchmark suite will benefit the entire community, including researchers, developers, hardware manufacturers, builders of machine learning frameworks, cloud service providers, application providers, and end users.
The STAC Benchmark Council consists of over 300 financial institutions and more than 50 vendor organizations whose purpose is to discuss technical challenges and solutions in financial services and to develop technology benchmark standards that are useful to financial organisations.
The Linked Data Benchmark Council (LDBC) was established as an outcome of the LDBC EU project funded by the European Commission within the 7th Framework Programme. Since the end of this project, it is a an independent non-profit company self-financed by membership dues.
The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse standardized benchmarks and tools to evaluate performance and energy efficiency for the newest generation of computing systems.
The TPC is a non-profit corporation founded to define transaction processing and database benchmarks and to disseminate objective, verifiable TPC performance data to the industry.
International Open Benchmarking Council (BenchCouncil) is a non-profit benchmarking organization, which aims to promote multi-disciplinary benchmarking research and practice and foster collaboration and interaction between industry and academia.
BigDataBench is an open-source big data benchmark suite. The current version BigDataBench 5.0 provides 13 representative real-world data sets and 27 big data benchmarks.