The dAIEdge-VLab pipeline is the set of steps that are executed to run the benchmarks on the different targets and to store the results in the blockchain. The pipeline is composed of several steps that are executed in a specific order. This page describes the different steps of the pipeline and the tools that are used to execute them.

Script execution

The main purpose of the pipeline is to execute the benchmark scripts defined in the docker image of the target repository. These scripts are the four mandatory scripts that are defined in the folder structure. These scripts are executed in the following order :

1. support.sh : This script is used to generate the code for the target. It is executed first to generate the code that will be used in the next steps. It is also used to setup the target and to prepare the environment for the next steps.
2. build.sh : This script is used to build the code for the target. It is executed after the support.sh script to build the code that was generated in the previous step.
3. deploy.sh : This script is used to deploy (program) the model on the target. It is executed after the build.sh script to deploy the code that was built in the previous step.
4. manager.sh : This script is used to gather metrics from the target and generate the benchmark report. It is executed after the deploy.sh script to gather metrics from the target and generate the benchmark report. The final report must be saved in the AI_Manager/out folder of the target repository.

Simulate the pipeline locally

You can simulate the pipeline locally by running the scripts in the correct order. This can be useful to test your implementation before pushing it to the repository and running it on the dAIEdge-VLab.

Environment variables

The variables defined in the target.json configuration file are passed to the container as environment variables. These variables are used to configure the execution of the pipeline and to provide the necessary information for the scripts to run. You can define these variables in the target.json file and use them in your scripts to make them more flexible and adaptable to different targets and configurations.

The dAIEdge-VLab also provides some additional environment variables that are automatically set when the scripts are executed. These variables provide information about the benchmark type, the model file, the dataset file, and other useful information that can be used in the implementation of the scripts. You can find the list of these additional environment variables in the environment variables documentation.

Error handling

The dAIEdge-VLab provides a way to handle errors and warnings in the scripts. The errors and user information are written to the error.log and user.log files respectively. The error.log file is used to write functional errors that may occur during the execution of the scripts, while the user.log file is used to write warnings and user information.

One core principle of the dAIEdge-VLab pipeline is that as soon as the error.log file is not empty, the dAIEdge-VLab stops the execution of the pipeline and considers that the benchmark has failed. Therefore, it is important to handle errors properly and to write them to the error.log file only if they are functional errors. For other types of errors, it is recommended to write them to the user.log file to avoid information leaks and to provide useful information to the user.