Skip to content
Developper Doc
Services
Benchmarking service with user data (TYPE2)

Benchmarking service with user data (TYPE2)

In this mode, the benchmark of the model is performed uesing the preprocessed dataset given by the user. This dataset is given in a binary format. The number of inference performed is based on the size of the dataset. The dataset file name is available in the variable $DATASET_FILENAME and is automatically placed in the root folder.

Usefull environment variables

The following environment variables are provided to the docker container running the target scripts. These are the only variables that are usefull for the implementation of the TYPE2 benchmarking service.

Variable name Description
$TARGET The target name
$RUNTIME The runtime name
$BENCHMARK_TYPE TYPE2
$MODEL_FILENAME Provide the file name of the model provided by the user. Note that a timestamp is added to it automatically to ensure uniqueness.
$DATASET_FILENAME Provide the file name of the dataset provided by the user. Only if one was provided.
$PARAMETERS_FILENAME Provide the file name of the parameters file containing parameters linked to the runtime. Only if one was provided.

Pipeline simulation

You can simulate the pipeline locally by running the scripts in the correct order. This can be useful to test your implementation before pushing it to the repository and running it on the dAIEdge-VLab.

export FUNCTIONS_PATH=`pwd`/exit_functions.sh
export LOG_PATH=`pwd`

export TARGET="jetsonorinnano_ort"
export RUNTIME="ORT"
export NB_INFERENCE=20
export BENCHMARK_TYPE="TYPE2"
export MODEL_FILENAME="mnist-12.onnx"
export PARAMETERS_FILENAME="parameters.json"
export DATASET_FILENAME="dataset_mnist.bin"
export FUNCTIONS_PATH=`pwd`
export LOG_PATH=`pwd`
export WORKSPACE_PATH=`pwd`


# Setup your own environment variables here if needed
#export BOARD_USER="jetsonorinnano"
#export BOARD_IP="192.168.2.20"
#export BOARD_PASSWORD="1234"


mkdir $LOG_PATH
touch $LOG_PATH/error.log
touch $LOG_PATH/user.log

exit_if_error_log_filled() {
    if [ -s "$LOG_PATH/error.log" ]; then
        echo "Errors detected in error.log. Exiting."
        exit 1
    fi
}

# Setup your own environment variables here if needed
#export BOARD_USER="agx"
#export BOARD_IP="157.26.100.91"
#export BOARD_PASSWORD="1234"

./AI_Support/support.sh;
exit_if_error_log_filled

./AI_Build/build.sh;
exit_if_error_log_filled

./AI_Deploy/deploy.sh;
exit_if_error_log_filled

./AI_Manager/manager.sh;
exit_if_error_log_filled

Generated artifacts

The following artifacts are generated by the dAIEdge-VLab after the execution of the pipeline:

Benchmarking service with random data (TYPE1)On-device Training (TYPE3)