Scientific benchmarking using OpenEBench

The scientific benchmarking process includes the following three steps:

1. Execution workflows: this activity correspond to the execution of the tools or workflows (participants) with some input datasets in order to get the predictions that are going to be used for the benchmarking. This can be done, for example, using Galaxy, Nextflow or Common Workflow Language.
2. Benchmarking workflow: this activity correspond to the benchmarking execution, this workflow includes the validation of the input data (output data coming from the participants execution), the computation of the metrics that are going to be used for the participants evaluation, and the consolidation of the results. During the validation, the input file format is checked and the content of the file is validated. Then, during the metrics computation, the predictions are compared with the reference datasets in order to evalaute the performance of each tool by different metrics. Finally, during the consolidation, the results of a particular participant are merged into the community’s data. 
3. Visualisation: the final activity of the bechmarking process is the long-term storage of benchmarking events and challenges to make the results of the performance of the tools avilable to be visualised graphically in the OpenEBench portal.

The process is represented in the following figure. The aim of this course is you are capable to execute a benchmarking workflow.