J6B Benchmark of Model Performance
-
Test Board: J6B
-
Range: Models under the
samples/ucp_tutorial/dnn/ai_benchmark/j6path in the OE package -
Runtime Environment: QNX
MODEL ACCURACY
| MODEL NAME | INPUT SIZE | ACCURACY | Dataset |
|---|---|---|---|
| ResNet50 | 1x3x224x224 | Top1: 0.7703(FLOAT)/0.7661(INT8) | ImageNet |
| GoogleNet | 1x3x224x224 | Top1: 0.7018(FLOAT)/0.6995(INT8) | ImageNet |
| EfficientNet_Lite1 | 1x240x240x3 | Top1: 0.7652(FLOAT)/0.7602(INT8) | ImageNet |
| EfficientNet_Lite2 | 1x260x260x3 | Top1: 0.7734(FLOAT)/0.7696(INT8) | ImageNet |
| EfficientNet_Lite3 | 1x280x280x3 | Top1: 0.7917(FLOAT)/0.7885(INT8) | ImageNet |
| EfficientNet_Lite4 | 1x300x300x3 | Top1: 0.8063(FLOAT)/0.8041(INT8) | ImageNet |
| Vargconvnet | 1x3x224x224 | Top1: 0.7793(FLOAT)/0.7765(INT8) | ImageNet |
| Efficientnasnet_m | 1x3x300x300 | Top1: 0.7935(FLOAT)/0.7923(INT8) | ImageNet |
| Efficientnasnet_s | 1x3x280x280 | Top1: 0.7441(FLOAT)/0.7516(INT8) | ImageNet |
| ResNet18 | 1x3x224x224 | Top1: 0.6976(FLOAT)/0.6948(INT8) | ImageNet |
| YOLOv2_Darknet19 | 1x3x608x608 | [IoU=0.50:0.95]= 0.2760(FLOAT)/0.2707(INT8) | COCO |
| YOLOv3_Darknet53 | 1x3x416x416 | [IoU=0.50:0.95]= 0.3370(FLOAT)/0.3370(INT8) | COCO |
| YOLOv5x_v2.0 | 1x3x672x672 | [IoU=0.50:0.95]= 0.4810(FLOAT)/0.4670(INT8) | COCO |
| Centernet_resnet101 | 1x3x512x512 | [IoU=0.50:0.95]= 0.3420(FLOAT)/0.3270(INT8) | COCO |
| YOLOv3_VargDarknet | 1x3x416x416 | [IoU=0.50:0.95]= 0.3280(FLOAT)/0.3260(INT8) | COCO |
| Deeplabv3plus_efficientnetb0 | 1x3x1024x2048 | mIoU: 0.7630(FLOAT)/0.7569(INT8) | Cityscapes |
| Fastscnn_efficientnetb0 | 1x3x1024x2048 | mIoU: 0.6997(FLOAT)/0.6909(INT8) | Cityscapes |
| Deeplabv3plus_efficientnetm1 | 1x3x1024x2048 | mIoU: 0.7794(FLOAT)/0.7756(INT8) | Cityscapes |
| Deeplabv3plus_efficientnetm2 | 1x3x1024x2048 | mIoU: 0.7882(FLOAT)/0.7854(INT8) | Cityscapes |
| Flashocc_henet_lss_occ3d_nuscenes | img: 6x3x512x960 points: 10x128x128x2 points_depth: 10x128x128x2 | mIoU: 0.3675(FLOAT)/0.3685(INT8) | Nuscenes |
| FCOS3D_efficientnetb0 | 1x3x512x896 | NDS: 0.3060(FLOAT)/0.3022(INT8) mAP: 0.2133(FLOAT)/0.2098(INT8) | nuscenes |
| Unet_mobilenetv1 | 1x3x1024x2048 | mIoU: 0.6802(FLOAT)/0.6764(INT8) | Cityscapes |
| Maptroe_henet_tinym_bevformer | img: 6x3x480x800 osm_mask: 1x1x50x100 queries_rebatch_grid: 6x20x100x2 restore_bev_grid: 1x100x100x2 reference_points_rebatch: 6x2000x4x2 bev_pillar_counts: 1x5000x1 | mAP: 0.6632(FLOAT)/0.6566(INT8) | Nuscenes |
MODEL PERFORMANCE
-
FPS = Frames per second. This data is obtained by running the
hrt_model_exectool in multi-threaded mode. For detailed usage instructions, please refer to the the hrt_model_exec tool introduction - model performance evaluation section. -
Latency = Inference latency, with the unit of ms(milliseconds). This data is obtained by running the
hrt_model_exectool in single-threaded mode. For detailed usage instructions, please refer to the the hrt_model_exec tool introduction - model performance evaluation section.
| MODEL NAME | INPUT SIZE | Latency(ms) | FPS | FPS Configuration |
|---|---|---|---|---|
| ResNet50 | 1x3x224x224 | 2.629 | 467.49 | thread_num:4 |
| GoogleNet | 1x3x224x224 | 1.325 | 1188.45 | thread_num:4 |
| EfficientNet_Lite1 | 1x240x240x3 | 1.195 | 1609.86 | thread_num:4 |
| EfficientNet_Lite2 | 1x260x260x3 | 1.517 | 1050.37 | thread_num:4 |
| EfficientNet_Lite3 | 1x280x280x3 | 1.875 | 769.68 | thread_num:4 |
| EfficientNet_Lite4 | 1x300x300x3 | 2.533 | 508.18 | thread_num:4 |
| Vargconvnet | 1x3x224x224 | 2.131 | 603.46 | thread_num:4 |
| Efficientnasnet_m | 1x3x300x300 | 2.211 | 578.34 | thread_num:4 |
| Efficientnasnet_s | 1x3x280x280 | 1.169 | 1449.07 | thread_num:4 |
| ResNet18 | 1x3x224x224 | 1.542 | 957.04 | thread_num:4 |
| YOLOv2_Darknet19 | 1x3x608x608 | 14.075 | 73.44 | thread_num:4 |
| YOLOv3_Darknet53 | 1x3x416x416 | 13.658 | 75.94 | thread_num:4 |
| YOLOv5x_v2.0 | 1x3x672x672 | 48.111 | 21.02 | thread_num:4 |
| Centernet_resnet101 | 1x3x512x512 | 15.086 | 68.53 | thread_num:4 |
| YOLOv3_VargDarknet | 1x3x416x416 | 9.491 | 111.18 | thread_num:4 |
| Deeplabv3plus_efficientnetb0 | 1x3x1024x2048 | 15.104 | 68.32 | thread_num:4 |
| Fastscnn_efficientnetb0 | 1x3x1024x2048 | 8.316 | 127.50 | thread_num:4 |
| Deeplabv3plus_efficientnetm1 | 1x3x1024x2048 | 27.896 | 36.42 | thread_num:4 |
| Deeplabv3plus_efficientnetm2 | 1x3x1024x2048 | 41.909 | 24.14 | thread_num:4 |
| Flashocc_henet_lss_occ3d_nuscenes | img: 6x3x512x960 points: 10x128x128x2 points_depth: 10x128x128x2 | 30.401 | 33.67 | thread_num:6 |
| FCOS3D_efficientnetb0 | 1x3x512x896 | 6.676 | 183.38 | thread_num:4 |
| Unet_mobilenetv1 | 1x3x1024x2048 | 3.842 | 327.50 | thread_num:4 |
| Maptroe_henet_tinym_bevformer | img: 6x3x480x800 osm_mask: 1x1x50x100 queries_rebatch_grid: 6x20x100x2 restore_bev_grid: 1x100x100x2 reference_points_rebatch: 6x2000x4x2 bev_pillar_counts: 1x5000x1 | 36.380 | 28.11 | thread_num:4 |