rpi5
Target information
| Description | Value |
|---|---|
| Number of instances | 5 |
| Number of app types | 2 |
| Number of engines | 3 |
Target Instances
App Type: TYPE1
Test Failed Across Instances
| Instance | ort | tflite | aidge |
|---|---|---|---|
| rpi5-0002-0000 | 1/22 | 1/23 | - |
| rpi5-0000-0002 | 1/22 | 1/23 | - |
| rpi5-0000-0000 | 1/22 | 1/23 | - |
| rpi5-0000-0001 | 1/22 | 1/23 | - |
| 0000019c71638acf3794… | 1/22 | 1/23 | 20/22 |
Inference Time Across Instances
Engine: ort
Inference Time: Significant variance in inference time across instances detected.
| Instance | efficientnet_b0_Opset16.onnx | efficientnet-lite4-11-int8.onnx | mobilenetv2-12-int8.onnx | mobilenetv2-12.onnx | mobilenetv3_large_100_miil_in21k_Opset17_timm.onnx | resnet50-v1-12-int8.onnx | dla60_res2net_Opset16.onnx | resnet34_Opset16.onnx | squeezenet1.0-12-int8.onnx | mnist-12-int8.onnx | mnist-7.onnx | bvlcalexnet-12-int8.onnx | tinyyolov2-8.onnx | ssd_mobilenet_v1_12-int8.onnx | ssd-12-int8.onnx | st_movenet_lightning_heatmaps_192.onnx | udnie-8.onnx | pointilism-9.onnx | candy-8.onnx | arcfaceresnet100-11-int8.onnx | age_googlenet.onnx | keypoint_classifier.tflite |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rpi5-0002-0000 | 312.75 ms | 115.45 ms | 29.85 ms | 140.10 ms | 173.89 ms | 89.99 ms | 279.31 ms | 189.60 ms | 20.24 ms | 0.93 ms | 0.57 ms | 60.36 ms | 502.82 ms | 450.82 ms | 2345.73 ms | 194.69 ms | 1480.53 ms | 1490.02 ms | 1494.39 ms | 783.01 ms | 237.65 ms | N/A |
| rpi5-0000-0002 | 314.05 ms | 118.75 ms | 30.31 ms | 142.96 ms | 174.81 ms | 90.62 ms | 275.43 ms | 177.05 ms | 20.59 ms | 0.93 ms | 0.56 ms | 60.20 ms | 428.10 ms | 354.21 ms | 1548.74 ms | 127.15 ms | 937.13 ms | 942.80 ms | 940.71 ms | 490.63 ms | 157.97 ms | N/A |
| rpi5-0000-0000 | 318.00 ms | 117.98 ms | 30.66 ms | 150.76 ms | 174.77 ms | 90.42 ms | 275.36 ms | 176.98 ms | 20.97 ms | 0.97 ms | 0.58 ms | 61.61 ms | 423.53 ms | 358.45 ms | 1647.94 ms | 127.98 ms | 941.15 ms | 937.43 ms | 944.81 ms | 495.36 ms | 153.70 ms | N/A |
| rpi5-0000-0001 | 316.94 ms | 118.33 ms | 32.51 ms | 141.62 ms | 175.50 ms | 92.02 ms | 275.13 ms | 179.37 ms | 20.85 ms | 0.96 ms | 0.57 ms | 60.09 ms | 428.04 ms | 359.77 ms | 1512.09 ms | 127.63 ms | 936.27 ms | 940.14 ms | 936.81 ms | 491.22 ms | 155.27 ms | N/A |
| 0000019c71638acf3794… | 315.90 ms | 118.40 ms | 32.08 ms | 141.84 ms | 177.85 ms | 90.91 ms | 285.26 ms | 185.59 ms | 20.92 ms | 0.94 ms | 0.58 ms | 60.81 ms | 461.64 ms | 368.00 ms | 2172.92 ms | 152.87 ms | 1327.36 ms | 1358.05 ms | 1351.64 ms | 704.94 ms | 209.37 ms | N/A |
Engine: tflite
Inference Time: Significant variance in inference time across instances detected.
| Instance | keypoint_classifier.tflite | st_efficientnet_lc_v1_128_tfs_int8.tflite | fdmobilenet_0.25_128_tfs_int8.tflite | st_fdmobilenet_v1_224_tfs_int8.tflite | mobilenet_v1_0.5_224_tfs_int8.tflite | mobilenet_v2_0.35_224_tl_int8.tflite | resnet_v1_32_32_tfs_int8.tflite | squeezenetv1.1_224_tfs_int8.tflite | miniresnet_2stacks_64x50_tl_int8.tflite | miniresnetv2_2stacks_64x50_tl_int8.tflite | yamnet_256_64x96_tl_int8.tflite | ssd_mobilenet_v2_fpnlite_100_416_int8.tflite | st_yolo_lc_v1_256_int8.tflite | tiny_yolo_v2_416_int8.tflite | YOLOv5.tflite | st_ssd_mobilenet_v1_025_256_int8.tflite | movenet_singlepose_lightning_192_float.tflite | movenet_singlepose_lightning_192_int8.tflite | movenet_lightning_heatmaps_256_int8_pt.tflite | movenet_singlepose_thunder_256_float.tflite | deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite | deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite | efficientnet_b0_Opset16.onnx |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rpi5-0002-0000 | 0.00 ms | 11.30 ms | 0.70 ms | 2.49 ms | 3.19 ms | 3.94 ms | 0.69 ms | 3.19 ms | 0.19 ms | 0.38 ms | 1.02 ms | 108.67 ms | 1.31 ms | 32.44 ms | 231.75 ms | 8.35 ms | 14.82 ms | 8.56 ms | 34.24 ms | 60.47 ms | 192.04 ms | 171.19 ms | N/A |
| rpi5-0000-0002 | 0.00 ms | 9.65 ms | 0.56 ms | 2.05 ms | 2.60 ms | 3.52 ms | 0.50 ms | 2.79 ms | 0.15 ms | 0.30 ms | 0.97 ms | 84.65 ms | 0.98 ms | 25.70 ms | 242.40 ms | 6.70 ms | 12.50 ms | 6.58 ms | 33.33 ms | 53.17 ms | 140.21 ms | 140.80 ms | N/A |
| rpi5-0000-0000 | 0.00 ms | 10.94 ms | 0.57 ms | 2.18 ms | 2.68 ms | 3.47 ms | 0.53 ms | 2.70 ms | 0.14 ms | 0.31 ms | 0.97 ms | 83.37 ms | 1.02 ms | 26.97 ms | 264.10 ms | 6.37 ms | 13.05 ms | 6.68 ms | 32.25 ms | 55.96 ms | 140.53 ms | 142.58 ms | N/A |
| rpi5-0000-0001 | 0.00 ms | 9.49 ms | 0.54 ms | 2.06 ms | 2.62 ms | 3.44 ms | 0.53 ms | 2.71 ms | 0.15 ms | 0.30 ms | 0.99 ms | 83.60 ms | 0.97 ms | 26.29 ms | 251.05 ms | 6.74 ms | 13.18 ms | 6.42 ms | 29.64 ms | 55.51 ms | 144.07 ms | 141.13 ms | N/A |
| 0000019c71638acf3794… | 0.00 ms | 8.76 ms | 0.63 ms | 2.08 ms | 2.77 ms | 3.60 ms | 0.54 ms | 2.81 ms | 0.15 ms | 0.31 ms | 1.00 ms | 86.16 ms | 1.06 ms | 27.81 ms | 263.98 ms | 5.96 ms | 13.19 ms | 6.47 ms | 30.97 ms | 55.60 ms | 153.75 ms | 154.39 ms | N/A |
Engine: aidge
Inference Time: Inference time across instances.
| Instance | efficientnet_b0_Opset16.onnx | efficientnet-lite4-11-int8.onnx | mobilenetv2-12-int8.onnx | mobilenetv2-12.onnx | mobilenetv3_large_100_miil_in21k_Opset17_timm.onnx | resnet50-v1-12-int8.onnx | dla60_res2net_Opset16.onnx | resnet34_Opset16.onnx | squeezenet1.0-12-int8.onnx | mnist-12-int8.onnx | mnist-7.onnx | bvlcalexnet-12-int8.onnx | tinyyolov2-8.onnx | ssd_mobilenet_v1_12-int8.onnx | ssd-12-int8.onnx | st_movenet_lightning_heatmaps_192.onnx | udnie-8.onnx | pointilism-9.onnx | candy-8.onnx | arcfaceresnet100-11-int8.onnx | age_googlenet.onnx | keypoint_classifier.tflite |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0000019c71638acf3794… | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | 2.56 ms | N/A | 1148.67 ms | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
App Type: TYPE2
Test Failed Across Instances
| Instance | tflite |
|---|---|
| rpi5-0002-0000 | 0/1 |
| rpi5-0000-0002 | 0/1 |
| rpi5-0000-0000 | 0/1 |
| rpi5-0000-0001 | 0/1 |
| 0000019c71638acf3794… | 0/1 |
Inference Time Across Instances
Engine: tflite
Inference Time: Inference time across instances.
| Instance | st_efficientnet_lc_v1_128_tfs_int8.tflite |
|---|---|
| rpi5-0002-0000 | 9.78 ms |
| rpi5-0000-0002 | 9.01 ms |
| rpi5-0000-0000 | 8.99 ms |
| rpi5-0000-0001 | 9.02 ms |
| 0000019c71638acf3794… | 9.01 ms |
Accuracy Across Instances
Engine: tflite
Accuracy: Accuracy across instances.
| Instance | st_efficientnet_lc_v1_128_tfs_int8.tflite |
|---|---|
| rpi5-0002-0000 | 99.87 % |
| rpi5-0000-0002 | 99.87 % |
| rpi5-0000-0000 | 99.87 % |
| rpi5-0000-0001 | 99.87 % |
| 0000019c71638acf3794… | 99.87 % |
Inference time across applications
st_efficientnet_lc_v1_128_tfs_int8.tflite
| Instance | TYPE1 tflite | TYPE2 tflite |
|---|---|---|
| 0000019c71638acf3794… | 11.30 ms | 9.78 ms |
| 0000019c71638acf3794… | 9.65 ms | 9.01 ms |
| 0000019c71638acf3794… | 10.94 ms | 8.99 ms |
| 0000019c71638acf3794… | 9.49 ms | 9.02 ms |
| 0000019c71638acf3794… | 8.76 ms | 9.01 ms |