J6B Benchmark of Model Performance
-
Test Board: J6B
-
Range: Models under the
samples/ucp_tutorial/dnn/ai_benchmark/j6path in the OE package -
Runtime Environment: QNX
MODEL ACCURACY
| MODEL NAME | INPUT SIZE | ACCURACY | Dataset |
|---|---|---|---|
| ResNet50 | 1x3x224x224 | Top1: 0.7703(FLOAT)/0.7661(INT8) | ImageNet |
| GoogleNet | 1x3x224x224 | Top1: 0.7018(FLOAT)/0.6995(INT8) | ImageNet |
| EfficientNet_Lite1 | 1x240x240x3 | Top1: 0.7652(FLOAT)/0.7602(INT8) | ImageNet |
| EfficientNet_Lite2 | 1x260x260x3 | Top1: 0.7734(FLOAT)/0.7696(INT8) | ImageNet |
| EfficientNet_Lite3 | 1x280x280x3 | Top1: 0.7917(FLOAT)/0.7885(INT8) | ImageNet |
| EfficientNet_Lite4 | 1x300x300x3 | Top1: 0.8063(FLOAT)/0.8041(INT8) | ImageNet |
| Vargconvnet | 1x3x224x224 | Top1: 0.7793(FLOAT)/0.7765(INT8) | ImageNet |
| Efficientnasnet_m | 1x3x300x300 | Top1: 0.7935(FLOAT)/0.7923(INT8) | ImageNet |
| Efficientnasnet_s | 1x3x280x280 | Top1: 0.7441(FLOAT)/0.7516(INT8) | ImageNet |
| ResNet18 | 1x3x224x224 | Top1: 0.6976(FLOAT)/0.6948(INT8) | ImageNet |
| YOLOv2_Darknet19 | 1x3x608x608 | [IoU=0.50:0.95]= 0.2760(FLOAT)/0.2707(INT8) | COCO |
| YOLOv3_Darknet53 | 1x3x416x416 | [IoU=0.50:0.95]= 0.3370(FLOAT)/0.3370(INT8) | COCO |
| YOLOv5x_v2.0 | 1x3x672x672 | [IoU=0.50:0.95]= 0.4810(FLOAT)/0.4670(INT8) | COCO |
| Centernet_resnet101 | 1x3x512x512 | [IoU=0.50:0.95]= 0.3420(FLOAT)/0.3270(INT8) | COCO |
| YOLOv3_VargDarknet | 1x3x416x416 | [IoU=0.50:0.95]= 0.3280(FLOAT)/0.3260(INT8) | COCO |
| Deeplabv3plus_efficientnetb0 | 1x3x1024x2048 | mIoU: 0.7630(FLOAT)/0.7569(INT8) | Cityscapes |
| Fastscnn_efficientnetb0 | 1x3x1024x2048 | mIoU: 0.6997(FLOAT)/0.6909(INT8) | Cityscapes |
| Deeplabv3plus_efficientnetm1 | 1x3x1024x2048 | mIoU: 0.7794(FLOAT)/0.7756(INT8) | Cityscapes |
| Deeplabv3plus_efficientnetm2 | 1x3x1024x2048 | mIoU: 0.7882(FLOAT)/0.7854(INT8) | Cityscapes |
| Flashocc_henet_lss_occ3d_nuscenes | img: 6x3x512x960 points: 10x128x128x2 points_depth: 10x128x128x2 | mIoU: 0.3675(FLOAT)/0.3685(INT8) | Nuscenes |
| FCOS3D_efficientnetb0 | 1x3x512x896 | NDS: 0.3061(FLOAT)/0.3022(INT8) mAP: 0.2133(FLOAT)/0.2098(INT8) | nuscenes |
| Unet_mobilenetv1 | 1x3x1024x2048 | mIoU: 0.6802(FLOAT)/0.6764(INT8) | Cityscapes |
| Maptroe_henet_tinym_bevformer | img: 6x3x480x800 osm_mask: 1x1x50x100 queries_rebatch_grid: 6x20x100x2 restore_bev_grid: 1x100x100x2 reference_points_rebatch: 6x2000x4x2 bev_pillar_counts: 1x5000x1 | mAP: 0.6632(FLOAT)/0.6566(INT8) | Nuscenes |
MODEL PERFORMANCE
-
FPS = Frames per second. This data is obtained by running the
hrt_model_exectool in multi-threaded mode. For detailed usage instructions, please refer to the the hrt_model_exec tool introduction - model performance evaluation section. -
Latency = Inference latency, with the unit of ms(milliseconds). This data is obtained by running the
hrt_model_exectool in single-threaded mode. For detailed usage instructions, please refer to the the hrt_model_exec tool introduction - model performance evaluation section.
| MODEL NAME | INPUT SIZE | Latency(ms) | FPS | FPS Configuration |
|---|---|---|---|---|
| ResNet50 | 1x3x224x224 | 2.585 | 469.11 | thread_num:4 |
| GoogleNet | 1x3x224x224 | 1.276 | 1204.82 | thread_num:4 |
| EfficientNet_Lite1 | 1x240x240x3 | 1.162 | 1617.32 | thread_num:4 |
| EfficientNet_Lite2 | 1x260x260x3 | 1.485 | 1053.13 | thread_num:4 |
| EfficientNet_Lite3 | 1x280x280x3 | 1.832 | 771.38 | thread_num:4 |
| EfficientNet_Lite4 | 1x300x300x3 | 2.496 | 510.08 | thread_num:4 |
| Vargconvnet | 1x3x224x224 | 2.104 | 604.94 | thread_num:4 |
| Efficientnasnet_m | 1x3x300x300 | 2.165 | 581.15 | thread_num:4 |
| Efficientnasnet_s | 1x3x280x280 | 1.135 | 1455.17 | thread_num:4 |
| ResNet18 | 1x3x224x224 | 1.497 | 961.40 | thread_num:4 |
| YOLOv2_Darknet19 | 1x3x608x608 | 14.025 | 73.55 | thread_num:4 |
| YOLOv3_Darknet53 | 1x3x416x416 | 13.587 | 76.07 | thread_num:4 |
| YOLOv5x_v2.0 | 1x3x672x672 | 48.007 | 21.04 | thread_num:4 |
| Centernet_resnet101 | 1x3x512x512 | 15.050 | 68.50 | thread_num:4 |
| YOLOv3_VargDarknet | 1x3x416x416 | 9.440 | 111.42 | thread_num:4 |
| Deeplabv3plus_efficientnetb0 | 1x3x1024x2048 | 14.994 | 68.63 | thread_num:4 |
| Fastscnn_efficientnetb0 | 1x3x1024x2048 | 8.293 | 128.03 | thread_num:4 |
| Deeplabv3plus_efficientnetm1 | 1x3x1024x2048 | 26.878 | 37.77 | thread_num:4 |
| Deeplabv3plus_efficientnetm2 | 1x3x1024x2048 | 40.843 | 24.75 | thread_num:4 |
| Flashocc_henet_lss_occ3d_nuscenes | img: 6x3x512x960 points: 10x128x128x2 points_depth: 10x128x128x2 | 29.690 | 34.30 | thread_num:6 |
| FCOS3D_efficientnetb0 | 1x3x512x896 | 6.397 | 184.55 | thread_num:4 |
| Unet_mobilenetv1 | 1x3x1024x2048 | 3.790 | 331.25 | thread_num:4 |
| Maptroe_henet_tinym_bevformer | img: 6x3x480x800 osm_mask: 1x1x50x100 queries_rebatch_grid: 6x20x100x2 restore_bev_grid: 1x100x100x2 reference_points_rebatch: 6x2000x4x2 bev_pillar_counts: 1x5000x1 | 35.709 | 28.50 | thread_num:4 |