Horizon-Torch-Samples is an algorithm tool based on the Pytorch and Pytorch plugin interfaces, which is an efficient and user-friendly algorithm toolkit for Horizon BPUs.

PyTorch, on which Horizon-Torch-Samples relies, is a tensor library optimized for deep learning by using GPUs and CPUs, which is now one of the most popular deep learning frameworks. The Pytorch plugin is a set of quantization algorithm tools developed based on Pytorch. Focusing on the implementation of quantization functions close to the computing platform, its quantization algorithms are deeply coupled with Horizon computing platforms, and the quantization models trained with this tool can be compiled and run normally on Horizon BPUs.

As the basic framework of algorithm package developed by Horizon Robotics, Horizon-Torch-Samples is open to all algorithm users, developers, and researchers. Its quantization training is closely related to the Horizon processors and contains a complete process: Floating point training --> QAT training --> Fixed-point transformation prediction --> Model check compilation (for Horizon BPU) --> On-board accuracy simulation verification. It also provides state-of-the-art (SOTA) deep-learning models for common image tasks including classification, detection, segmentation, etc.

#Features

• Based on Pytorch and horizon_plugin_pytorch.
• Include a complete process from Floating point training to On-board accuracy simulation verification.
• Include SOTA models for common image tasks such as classification, detection, and segmentation. All samples are compatible with Horizon BPUs.

#Sample Models

Horizon-Torch-Samples currently includes the following deep learning models:

Classification Model

• mobilenetv1_imagenet
• mobilenetv2_imagenet
• resnet18_imagenet
• resnet50_imagenet
• vargnetv2_imagenet
• efficientnet_imagenet
• horizon_swin_transformer_imagenet
• mixvargenet_imagenet
• efficientnasnetm_imagenet
• efficientnasnets_imagenet
• vit_small_imagenet
• henet_tinye_imagenet
• henet_tinym_imagenet

Detection model

• fcos_efficientnetb0_mscoco
• fcos_efficientnetb1_mscoco
• fcos_efficientnetb2_mscoco
• fcos_efficientnetb3_mscoco
• detr_resnet50_mscoco
• detr_efficientnetb3_mscoco
• deform_detr_resnet50_mscoco
• fcos3d_efficientnetb0_nuscenes
• pointpillars_kitti_car
• centerpoint_pointpillar_nuscenes

Segmentation model

• deeplabv3plus_efficientnetm0_cityscapes
• deeplabv3plus_efficientnetm1_cityscapes
• deeplabv3plus_efficientnetm2_cityscapes
• fastscnn_efficientnetb0tiny_cityscapes
• unet_mobilenetv1_cityscapes

Lane detection model

• ganet_mixvargenet_culane

Multiple Object Track

• motr_efficientnetb3_mot17

Binocular Depth Estimation

• stereonetplus_mixvargenet_sceneflow

Bev Multi-task Model

• bev_ipm_efficientnetb0_multitask_nuscenes
• bev_lss_efficientnetb0_multitask_nuscenes
• bev_gkt_mixvargenet_multitask_nuscenes
• bev_ipm_4d_efficientnetb0_multitask_nuscenes
• detr3d_efficientnetb3_nuscenes
• petr_efficientnetb3_nuscenes
• bevformer_tiny_resnet50_detection_nuscenes
• bev_cft_efficientnetb3_nuscenes
• bev_sparse_henet_tinym_nuscenes

Keypoints Detection Model

• keypoint_efficientnetb0_carfusion

Lidar Multi-task Model

• centerpoint_mixvargnet_multitask_nuscenes

Trajectory Prediction Model

• densetnt_vectornet_argoverse1
• qcnet_oe_argoverse2

Occupancy Prediction Model

• flashocc_henet_lss_occ3d_nuscenes

Online Map Construction

• maptroe_henet_tinym_bevformer_nuscenes
• maptroe_sparse_henet_tinym_nuscenes

Lidar Fusion Bev Multi-task Model

• bevfusion_pointpillar_henet_multisensor_multitask_nuscenes
• bev_sparse_lidar_fusion_henet_tinym_nuscenes

In the above model, resnet18_imagenet, resnet50_imagenet, vargnetv2_imagenet, efficientnasnetm_imagenet, efficientnasnets_imagenet, efficientnet_imagenet, mixvargenet_imagenet, vargnetv2_imagenet, ganet_mixvargenet_culane, deeplabv3plus_efficientnetm0_cityscapes, deeplabv3plus_efficientnetm1_cityscapes, deeplabv3plus_efficientnetm2_cityscapes, fastscnn_efficientnetb0tiny_cityscapes, bev_gkt_mixvargenet_multitask_nuscenes, bev_ipm_efficientnetb0_multitask_nuscenes, bev_lss_efficientnetb0_multitask_nuscenes, flashocc_henet_lss_occ3d_nuscenes, detr3d_efficientnetb3_nuscenes and keypoint_efficientnetb0_carfusion only need to do calibration quantization accuracy to achieve the goal, detailed accuracy reference model_zoo.