• English
  • 3.8.1

    • J6P/H Benchmark of Model Performance

    • Test Board: J6P

    • Range: Models under the samples/ucp_tutorial/dnn/ai_benchmark/j6 path in the OE package

    • Runtime Environment: Linux

    MODEL ACCURACY

    MODEL NAMEINPUT SIZEACCURACYDataset
    ResNet50

    1x3x224x224

    Top1:

    0.7703(FLOAT)/0.7661(INT8)

    		ImageNet
    GoogleNet

    1x3x224x224

    Top1:

    0.7018(FLOAT)/0.6995(INT8)

    		ImageNet
    EfficientNet_Lite1

    1x240x240x3

    Top1:

    0.7652(FLOAT)/0.7602(INT8)

    		ImageNet
    EfficientNet_Lite2

    1x260x260x3

    Top1:

    0.7734(FLOAT)/0.7696(INT8)

    		ImageNet
    EfficientNet_Lite3

    1x280x280x3

    Top1:

    0.7917(FLOAT)/0.7885(INT8)

    		ImageNet
    EfficientNet_Lite4

    1x300x300x3

    Top1:

    0.8063(FLOAT)/0.8041(INT8)

    		ImageNet
    Vargconvnet

    1x3x224x224

    Top1:

    0.7793(FLOAT)/0.7765(INT8)

    		ImageNet
    Efficientnasnet_m

    1x3x300x300

    Top1:

    0.7935(FLOAT)/0.7923(INT8)

    		ImageNet
    Efficientnasnet_s

    1x3x280x280

    Top1:

    0.7441(FLOAT)/0.7516(INT8)

    		ImageNet
    ResNet18

    1x3x224x224

    Top1:

    0.6976(FLOAT)/0.6948(INT8)

    		ImageNet
    YOLOv2_Darknet19

    1x3x608x608

    [IoU=0.50:0.95]=

    0.2760(FLOAT)/0.2707(INT8)

    		COCO
    YOLOv3_Darknet53

    1x3x416x416

    [IoU=0.50:0.95]=

    0.3370(FLOAT)/0.3370(INT8)

    		COCO
    YOLOv5x_v2.0

    1x3x672x672

    [IoU=0.50:0.95]=

    0.4810(FLOAT)/0.4670(INT8)

    		COCO
    Centernet_resnet101

    1x3x512x512

    [IoU=0.50:0.95]=

    0.3420(FLOAT)/0.3270(INT8)

    		COCO
    YOLOv3_VargDarknet

    1x3x416x416

    [IoU=0.50:0.95]=

    0.3280(FLOAT)/0.3260(INT8)

    		COCO
    Deeplabv3plus_efficientnetb0

    1x3x1024x2048

    mIoU:

    0.7630(FLOAT)/0.7569(INT8)

    		Cityscapes
    Fastscnn_efficientnetb0

    1x3x1024x2048

    mIoU:

    0.6997(FLOAT)/0.6909(INT8)

    		Cityscapes
    Deeplabv3plus_efficientnetm1

    1x3x1024x2048

    mIoU:

    0.7794(FLOAT)/0.7756(INT8)

    		Cityscapes
    Deeplabv3plus_efficientnetm2

    1x3x1024x2048

    mIoU:

    0.7882(FLOAT)/0.7854(INT8)

    		Cityscapes
    Bev_lss_efficientnetb0_multitask

    image:

    6x3x256x704

    points(0&1):

    10x128x128x2

    NDS:

    0.3005(FLOAT)/0.3010(INT8)

    MeanIOU:

    0.5180(FLOAT)/0.5172(INT8)

    mAP:

    0.2062(FLOAT)/0.2047(INT8)

    		Nuscenes
    Detr3d_efficientnetb3

    coords(0-3):

    6x4x256x2

    image:

    6x3x512x1408

    masks:

    1x4x256x24

    NDS:

    0.3304(FLOAT)/0.3306(INT8)

    mAP:

    0.2753(FLOAT)/0.2742(INT8)

    		Nuscenes
    Petr_efficientnetb3

    image:

    6x3x512x1408

    pos_embed:

    1x96x44x256

    NDS:

    0.3765(FLOAT)/0.3748(INT8)

    mAP:

    0.3038(FLOAT)/0.2942(INT8)

    		Nuscenes
    Bevformer_tiny_resnet50_detection

    img:

    6x3x480x800

    prev_bev:

    1x2500x256

    prev_bev_ref:

    1x50x50x2

    queries_rebatch_grid:

    6x20x32x2

    restore_bev_grid:

    1x100x50x2

    reference_points_rebatch:

    6x640x4x2

    bev_pillar_counts:

    1x2500x1

    NDS:

    0.3713(FLOAT)/0.3701(INT8)

    mAP:

    0.2673(FLOAT)/0.2641(INT8)

    		Nuscenes
    Flashocc_henet_lss_occ3d_nuscenes

    img:

    6x3x512x960

    points:

    10x128x128x2

    points_depth:

    10x128x128x2

    mIoU:

    0.3675(FLOAT)/0.3685(INT8)

    		Nuscenes
    Centerpoint_pointpillar

    points:

    300000x5

    voxel_feature:

    1x5x20x40000

    coors:

    40000x4

    NDS:

    0.5832(FLOAT)/0.5820(INT8)

    mAP:

    0.4804(FLOAT)/0.4783(INT8)

    		Nuscenes
    FCOS3D_efficientnetb0

    1x3x512x896

    NDS:

    0.3061(FLOAT)/0.3022(INT8)

    mAP:

    0.2133(FLOAT)/0.2098(INT8)

    		nuscenes
    Ganet_mixvargenet

    1x3x320x800

    F1Score:

    0.7949(FLOAT)/0.7884(INT8)

    		CuLane
    Deformable_detr_resnet50

    1x3x800x1333

    [IoU=0.50:0.95]=

    0.4413(FLOAT)/0.4499(INT8)

    		MS COCO
    Unet_mobilenetv1

    1x3x1024x2048

    mIoU:

    0.6802(FLOAT)/0.6764(INT8)

    		Cityscapes
    Maptroe_henet_tinym_bevformer

    img:

    6x3x480x800

    osm_mask:

    1x1x50x100

    queries_rebatch_grid:

    6x20x100x2

    restore_bev_grid:

    1x100x100x2

    reference_points_rebatch:

    6x2000x4x2

    bev_pillar_counts:

    1x5000x1

    mAP:

    0.6632(FLOAT)/0.6566(INT8)

    		Nuscenes
    Qcnet_oe

    valid_mask:

    1x30x10

    valid_mask_a2a:

    1x10x30x30

    agent_type:

    1x30x1

    x_a_cur:

    1x1x30x1,1x1x30x1,1x1x30x1,1x1x30x1

    r_pl2a_cur:

    1x1x30x80,1x1x30x80,1x1x30x80

    r_t_cur:

    1x1x30x6,1x1x30x6,1x1x30x6,1x1x30x6

    r_a2a_cur:

    1x1x30x30,1x1x30x30,1x1x30x30

    x_a_mid_emb:

    1x30x2x128

    x_a:

    1x30x6x128

    pl_type,is_intersection:

    1x80

    r_pl2pl:

    1x1x80x80,1x1x80x80,1x1x80x80

    r_pt2pl:

    1x1x80x50,1x1x80x50,1x1x80x50

    mask_pl2pl:

    1x80x80

    magnitude,pt_type,side,mask:

    1x80x50

    mask_a2m:

    1x30x30

    mask_dst:

    1x30x1

    type_pl2pl:

    1x80x80

    hitrate:

    0.8026(FLOAT)/0.7979(INT8)

    		Argoverse 2

    MODEL PERFORMANCE

    Note

    Before performance evaluation, please refer to the following method to set the number of threads in the thread pool for CPU operator inference in model inference to 12 via environment variables:

    export HB_UCP_ENABLE_CPU_BACKEND_CORE_NUM=12
    MODEL NAMEINPUT SIZELatency(ms)FPSFPS Configuration
    ResNet50

    1x3x224x224

    		0.5826553.92thread_num:10
    GoogleNet

    1x3x224x224

    		0.38118761.00thread_num:12
    EfficientNet_Lite1

    1x240x240x3

    		0.38418139.78thread_num:12
    EfficientNet_Lite2

    1x260x260x3

    		0.44714117.23thread_num:10
    EfficientNet_Lite3

    1x280x280x3

    		0.51311575.29thread_num:10
    EfficientNet_Lite4

    1x300x300x3

    		0.6538334.94thread_num:10
    Vargconvnet

    1x3x224x224

    		0.52112421.03thread_num:12
    Efficientnasnet_m

    1x3x300x300

    		0.5819270.39thread_num:10
    Efficientnasnet_s

    1x3x280x280

    		0.37619655.71thread_num:12
    ResNet18

    1x3x224x224

    		0.38914388.70thread_num:12
    YOLOv2_Darknet19

    1x3x608x608

    		2.2241988.29thread_num:8
    YOLOv3_Darknet53

    1x3x416x416

    		2.4411765.76thread_num:8
    YOLOv5x_v2.0

    1x3x672x672

    		8.190495.13thread_num:8
    Centernet_resnet101

    1x3x512x512

    		2.4391793.76thread_num:8
    YOLOv3_VargDarknet

    1x3x416x416

    		1.6252696.32thread_num:8
    Deeplabv3plus_efficientnetb0

    1x3x1024x2048

    		2.9541468.90thread_num:8
    Fastscnn_efficientnetb0

    1x3x1024x2048

    		1.7392647.01thread_num:8
    Deeplabv3plus_efficientnetm1

    1x3x1024x2048

    		5.123817.97thread_num:8
    Deeplabv3plus_efficientnetm2

    1x3x1024x2048

    		7.326562.40thread_num:8
    Bev_lss_efficientnetb0_multitask

    image:

    6x3x256x704

    points(0&1):

    10x128x128x2

    		3.0561452.42thread_num:8
    Detr3d_efficientnetb3

    coords(0-3):

    6x4x256x2

    image:

    6x3x512x1408

    masks:

    1x4x256x24

    		15.104257.70thread_num:8
    Petr_efficientnetb3

    image:

    6x3x512x1408

    pos_embed:

    1x96x44x256

    		21.426188.17thread_num:8
    Bevformer_tiny_resnet50_detection

    img:

    6x3x480x800

    prev_bev:

    1x2500x256

    prev_bev_ref:

    1x50x50x2

    queries_rebatch_grid:

    6x20x32x2

    restore_bev_grid:

    1x100x50x2

    reference_points_rebatch:

    6x640x4x2

    bev_pillar_counts:

    1x2500x1

    		14.104276.96thread_num:8
    Flashocc_henet_lss_occ3d_nuscenes

    img:

    6x3x512x960

    points:

    10x128x128x2

    points_depth:

    10x128x128x2

    		5.506744.62thread_num:8
    Centerpoint_pointpillar

    points:

    300000x5

    voxel_feature:

    1x5x20x40000

    coors:

    40000x4

    		7.4811079.91thread_num:16
    FCOS3D_efficientnetb0

    1x3x512x896

    		1.4663593.00thread_num:10
    Ganet_mixvargenet

    1x3x320x800

    		0.55112404.70thread_num:12
    Deformable_detr_resnet50

    1x3x800x1333

    		77.00129.32thread_num:8
    Unet_mobilenetv1

    1x3x1024x2048

    		0.8846399.42thread_num:10
    Maptroe_henet_tinym_bevformer

    img:

    6x3x480x800

    osm_mask:

    1x1x50x100

    queries_rebatch_grid:

    6x20x100x2

    restore_bev_grid:

    1x100x100x2

    reference_points_rebatch:

    6x2000x4x2

    bev_pillar_counts:

    1x5000x1

    		6.779583.17thread_num:8
    Qcnet_oe

    valid_mask:

    1x30x10

    valid_mask_a2a:

    1x10x30x30

    agent_type:

    1x30x1

    x_a_cur:

    1x1x30x1,1x1x30x1,1x1x30x1,1x1x30x1

    r_pl2a_cur:

    1x1x30x80,1x1x30x80,1x1x30x80

    r_t_cur:

    1x1x30x6,1x1x30x6,1x1x30x6,1x1x30x6

    r_a2a_cur:

    1x1x30x30,1x1x30x30,1x1x30x30

    x_a_mid_emb:

    1x30x2x128

    x_a:

    1x30x6x128

    pl_type,is_intersection:

    1x80

    r_pl2pl:

    1x1x80x80,1x1x80x80,1x1x80x80

    r_pt2pl:

    1x1x80x50,1x1x80x50,1x1x80x50

    mask_pl2pl:

    1x80x80

    magnitude,pt_type,side,mask:

    1x80x50

    mask_a2m:

    1x30x30

    mask_dst:

    1x30x1

    type_pl2pl:

    1x80x80

    		2.6021653.80thread_num:8