Target information

Description	Value
Number of instances	4
Number of app types	2
Number of engines	3

Target Instances

0000019c716353c48a919c2708c2e96740c0117e3b39e93019ec189e217f7302

rpi4b-0000-0000

0000019c716362ba976cf0240430d8609d2545907baf474a564f4d508e331fdf

0000019c71635d3019f6027236bbb385337cac91fb67191971047760add7d2ef

App Type: TYPE1

Test Failed Across Instances

Instance	ort	aidge	tflite
0000019c716353c48a91…	1/22	20/22	1/23
rpi4b-0000-0000	1/22	-	1/23
0000019c716362ba976c…	1/22	-	1/23
0000019c71635d3019f6…	1/22	-	1/23

Inference Time Across Instances

Engine: ort

Instance	efficientnet_b0_Opset16.onnx	efficientnet-lite4-11-int8.onnx	mobilenetv2-12-int8.onnx	mobilenetv2-12.onnx	mobilenetv3_large_100_miil_in21k_Opset17_timm.onnx	resnet50-v1-12-int8.onnx	dla60_res2net_Opset16.onnx	resnet34_Opset16.onnx	squeezenet1.0-12-int8.onnx	mnist-12-int8.onnx	mnist-7.onnx	bvlcalexnet-12-int8.onnx	tinyyolov2-8.onnx	ssd_mobilenet_v1_12-int8.onnx	ssd-12-int8.onnx	st_movenet_lightning_heatmaps_192.onnx	udnie-8.onnx	pointilism-9.onnx	candy-8.onnx	arcfaceresnet100-11-int8.onnx	age_googlenet.onnx	keypoint_classifier.tflite
0000019c716353c48a91…	186.67 ms	106.50 ms	32.76 ms	94.01 ms	112.79 ms	182.94 ms	536.69 ms	439.00 ms	26.67 ms	0.58 ms	0.37 ms	56.86 ms	372.37 ms	269.09 ms	8240.75 ms	79.69 ms	996.39 ms	992.42 ms	1002.93 ms	698.83 ms	205.96 ms	N/A
rpi4b-0000-0000	204.51 ms	109.11 ms	40.26 ms	99.65 ms	124.38 ms	193.50 ms	912.06 ms	434.17 ms	28.91 ms	0.56 ms	1.32 ms	60.29 ms	462.99 ms	279.19 ms	6864.23 ms	86.36 ms	1006.18 ms	1014.07 ms	987.71 ms	639.76 ms	214.58 ms	N/A
0000019c716362ba976c…	194.58 ms	100.90 ms	35.41 ms	97.27 ms	118.16 ms	184.28 ms	536.72 ms	416.44 ms	26.43 ms	0.60 ms	0.83 ms	89.82 ms	382.18 ms	270.09 ms	6960.38 ms	84.58 ms	1007.85 ms	972.82 ms	964.49 ms	638.65 ms	209.02 ms	N/A
0000019c71635d3019f6…	203.94 ms	109.99 ms	40.12 ms	103.68 ms	120.02 ms	193.09 ms	552.31 ms	629.14 ms	29.87 ms	0.55 ms	0.39 ms	62.03 ms	531.71 ms	281.39 ms	7165.81 ms	89.10 ms	1002.29 ms	1207.40 ms	1095.11 ms	650.04 ms	218.96 ms	N/A

Engine: aidge

Instance	efficientnet_b0_Opset16.onnx	efficientnet-lite4-11-int8.onnx	mobilenetv2-12-int8.onnx	mobilenetv2-12.onnx	mobilenetv3_large_100_miil_in21k_Opset17_timm.onnx	resnet50-v1-12-int8.onnx	dla60_res2net_Opset16.onnx	resnet34_Opset16.onnx	squeezenet1.0-12-int8.onnx	mnist-12-int8.onnx	mnist-7.onnx	bvlcalexnet-12-int8.onnx	tinyyolov2-8.onnx	ssd_mobilenet_v1_12-int8.onnx	ssd-12-int8.onnx	st_movenet_lightning_heatmaps_192.onnx	udnie-8.onnx	pointilism-9.onnx	candy-8.onnx	arcfaceresnet100-11-int8.onnx	age_googlenet.onnx	keypoint_classifier.tflite
0000019c716353c48a91…	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	4.38 ms	N/A	3248.45 ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A

Engine: tflite

Instance	keypoint_classifier.tflite	st_efficientnet_lc_v1_128_tfs_int8.tflite	fdmobilenet_0.25_128_tfs_int8.tflite	st_fdmobilenet_v1_224_tfs_int8.tflite	mobilenet_v1_0.5_224_tfs_int8.tflite	mobilenet_v2_0.35_224_tl_int8.tflite	resnet_v1_32_32_tfs_int8.tflite	squeezenetv1.1_224_tfs_int8.tflite	miniresnet_2stacks_64x50_tl_int8.tflite	miniresnetv2_2stacks_64x50_tl_int8.tflite	yamnet_256_64x96_tl_int8.tflite	ssd_mobilenet_v2_fpnlite_100_416_int8.tflite	st_yolo_lc_v1_256_int8.tflite	tiny_yolo_v2_416_int8.tflite	YOLOv5.tflite	st_ssd_mobilenet_v1_025_256_int8.tflite	movenet_singlepose_lightning_192_float.tflite	movenet_singlepose_lightning_192_int8.tflite	movenet_lightning_heatmaps_256_int8_pt.tflite	movenet_singlepose_thunder_256_float.tflite	deeplab_v3_mobilenetv2_05_16_512_asppv1_int8.tflite	deeplab_v3_mobilenetv2_05_16_512_asppv1_int8_f32.tflite	efficientnet_b0_Opset16.onnx
0000019c716353c48a91…	0.01 ms	24.13 ms	1.48 ms	5.45 ms	20.50 ms	19.73 ms	6.21 ms	21.87 ms	1.65 ms	3.17 ms	3.91 ms	196.54 ms	9.88 ms	252.93 ms	343.84 ms	15.76 ms	47.14 ms	43.95 ms	103.26 ms	177.28 ms	306.94 ms	297.18 ms	N/A
rpi4b-0000-0000	0.01 ms	23.97 ms	1.60 ms	5.22 ms	20.31 ms	20.36 ms	5.87 ms	22.02 ms	1.67 ms	3.41 ms	4.13 ms	209.92 ms	10.48 ms	256.48 ms	367.19 ms	15.64 ms	53.87 ms	45.37 ms	108.86 ms	189.62 ms	319.41 ms	310.55 ms	N/A
0000019c716362ba976c…	0.00 ms	23.81 ms	1.65 ms	5.51 ms	10.03 ms	6.81 ms	3.53 ms	16.04 ms	1.65 ms	2.88 ms	1.28 ms	206.09 ms	6.11 ms	172.93 ms	342.19 ms	15.39 ms	51.03 ms	22.87 ms	70.69 ms	174.87 ms	303.54 ms	304.22 ms	N/A
0000019c71635d3019f6…	0.02 ms	23.29 ms	1.52 ms	5.79 ms	19.83 ms	20.24 ms	6.03 ms	21.60 ms	1.84 ms	3.08 ms	4.45 ms	202.96 ms	10.35 ms	267.63 ms	369.32 ms	15.98 ms	55.32 ms	45.64 ms	108.09 ms	188.59 ms	315.85 ms	309.51 ms	N/A

App Type: TYPE2

Test Failed Across Instances

Instance	tflite
0000019c716353c48a91…	0/1
rpi4b-0000-0000	0/1
0000019c716362ba976c…	0/1
0000019c71635d3019f6…	0/1

Inference Time Across Instances

Engine: tflite

Instance	st_efficientnet_lc_v1_128_tfs_int8.tflite
0000019c716353c48a91…	21.01 ms
rpi4b-0000-0000	21.00 ms
0000019c716362ba976c…	22.03 ms
0000019c71635d3019f6…	21.00 ms

Accuracy Across Instances

Engine: tflite

Instance	st_efficientnet_lc_v1_128_tfs_int8.tflite
0000019c716353c48a91…	99.87 %
rpi4b-0000-0000	99.87 %
0000019c716362ba976c…	99.87 %
0000019c71635d3019f6…	99.87 %

Inference time across applications

st_efficientnet_lc_v1_128_tfs_int8.tflite

Instance	TYPE1 tflite	TYPE2 tflite
0000019c71635d3019f6…	24.13 ms	21.01 ms
0000019c71635d3019f6…	23.97 ms	21.00 ms
0000019c71635d3019f6…	23.81 ms	22.03 ms
0000019c71635d3019f6…	23.29 ms	21.00 ms