RelBench: Relational Deep Learning Benchmark

Open benchmark for machine learning over relational databases

The Relational Deep Learning Benchmark (RelBench) is a collection of realistic, large-scale, and diverse benchmark datasets for machine learning on relational databases. RelBench datasets are automatically downloaded, processed, and split using the Data Loader. The model performance can be evaluated using the Evaluator in a unified manner. RelBench is a community-driven initiative in active development. We expect the benchmark datasets to evolve.

RelBench is currently in its beta testing phase, stay tuned for more updates!

Realistic Databases

RelBench provides a diverse set of challenging and realistic benchmark relational databases and predictive tasks that are of varying sizes and fields.

Flexible Data Loaders

RelBench fully automates processing over relational databases. It will download and process databases, provide graph objects that are fully compatible with Pytorch Geometric.

Evaluators

RelBench provides unified dataset splits and evaluators that allow for easy and reliable comparison of different models in a unified manner. RelBench uses leaderboards to keep track of the state-of-the-art.

Get Started