Horizon-Torch-Samples is an algorithm tool based on the Pytorch and Pytorch plugin interfaces, which is an efficient and user-friendly algorithm toolkit for Horizon BPUs.

PyTorch, on which Horizon-Torch-Samples relies, is a tensor library optimized for deep learning by using GPUs and CPUs, which is now one of the most popular deep learning frameworks. The Pytorch plugin is a set of quantization algorithm tools developed based on Pytorch. Focusing on the implementation of quantization functions close to the computing platform, its quantization algorithms are deeply coupled with Horizon computing platforms, and the quantization models trained with this tool can be compiled and run normally on Horizon BPUs.

As the basic framework of algorithm package developed by Horizon Robotics, Horizon-Torch-Samples is open to all algorithm users, developers, and researchers. Its quantization training is closely related to the Horizon processors and contains a complete process: Floating point training --> QAT training --> Fixed-point transformation prediction --> Model check compilation (for Horizon BPU) --> On-board accuracy simulation verification. It also provides state-of-the-art (SOTA) deep-learning models for common image tasks including classification, detection, segmentation, etc.

#Features

• Based on Pytorch and horizon_plugin_pytorch.
• Include a complete process from Floating point training to On-board accuracy simulation verification.
• Include SOTA models for common image tasks such as classification, detection, and segmentation. All samples are compatible with Horizon BPUs.

#Sample Models

Horizon-Torch-Samples currently includes the following deep learning models:

Model Category	Model Name
Classification Model	mobilenetv1_imagenet mobilenetv2_imagenet resnet18_imagenet resnet50_imagenet vargnetv2_imagenet efficientnet_imagenet horizon_swin_transformer_imagenet mixvargenet_imagenet efficientnasnetm_imagenet efficientnasnets_imagenet vit_small_imagenet henet_tinye_imagenet henet_tinym_imagenet
Detection model	fcos_efficientnetb0_mscoco fcos_efficientnetb1_mscoco fcos_efficientnetb2_mscoco fcos_efficientnetb3_mscoco detr_resnet50_mscoco detr_efficientnetb3_mscoco deform_detr_resnet50_mscoco fcos3d_efficientnetb0_nuscenes pointpillars_kitti_car centerpoint_pointpillar_nuscenes
Segmentation model	deeplabv3plus_efficientnetm0_cityscapes deeplabv3plus_efficientnetm1_cityscapes deeplabv3plus_efficientnetm2_cityscapes fastscnn_efficientnetb0tiny_cityscapes unet_mobilenetv1_cityscapes
Lane detection model	ganet_mixvargenet_culane
Multiple Object Track	motr_efficientnetb3_mot17
Binocular Depth Estimation	stereonetplus_mixvargenet_sceneflow
Bev Multi-task Model	bev_ipm_efficientnetb0_multitask_nuscenes bev_lss_efficientnetb0_multitask_nuscenes bev_gkt_mixvargenet_multitask_nuscenes bev_ipm_4d_efficientnetb0_multitask_nuscenes detr3d_efficientnetb3_nuscenes petr_efficientnetb3_nuscenes bevformer_tiny_resnet50_detection_nuscenes bev_cft_efficientnetb3_nuscenes bev_sparse_henet_tinym_nuscenes bev_sparse_det_maptr_flashocc_henet_tinym_nuscenes
Keypoints Detection Model	keypoint_efficientnetb0_carfusion
Lidar Multi-task Model	centerpoint_mixvargnet_multitask_nuscenes
Trajectory Prediction Model	densetnt_vectornet_argoverse1 qcnet_oe_argoverse2
Occupancy Prediction Model	flashocc_henet_lss_occ3d_nuscenes
Online Map Construction	maptroe_henet_tinym_bevformer_nuscenes maptroe_sparse_henet_tinym_nuscenes
Lidar Fusion Bev Multi-task Model	bevfusion_pointpillar_henet_multisensor_multitask_nuscenes bev_sparse_lidar_fusion_henet_tinym_nuscenes

In the above model, resnet18_imagenet, resnet50_imagenet, vargnetv2_imagenet, efficientnasnetm_imagenet, efficientnasnets_imagenet, efficientnet_imagenet, mixvargenet_imagenet, vargnetv2_imagenet, ganet_mixvargenet_culane, deeplabv3plus_efficientnetm0_cityscapes, deeplabv3plus_efficientnetm1_cityscapes, deeplabv3plus_efficientnetm2_cityscapes, fastscnn_efficientnetb0tiny_cityscapes, bev_gkt_mixvargenet_multitask_nuscenes, bev_ipm_efficientnetb0_multitask_nuscenes, bev_lss_efficientnetb0_multitask_nuscenes, bev_sparse_det_maptr_flashocc_henet_tinym_nuscenes, flashocc_henet_lss_occ3d_nuscenes, detr3d_efficientnetb3_nuscenes and keypoint_efficientnetb0_carfusion only need to do calibration quantization accuracy to achieve the goal, detailed accuracy reference model_zoo.