Common Algorithm Model Samples

Sample Location

The common algorithmic model samples are located at classification/, detection/ and segmentation/ folders in the onnx_ptq/examples path.

Prepare Datasets

Dataset Download Address

The dataset can be downloaded from the following address.

Dataset	Address
ImageNet	https://www.image-net.org/download.php
COCO	https://cocodataset.org/
VOC	http://host.robots.ox.ac.uk/pascal/VOC/ (need to download both versions 2007 and 2012)
Cityscapes	https://github.com/mcordts/cityscapesScripts

Dataset Reference Structure

To facilitate your subsequent steps, after the dataset has been downloaded, you need to process the evaluation dataset according to the structure suggested by Horizon below.

Dataset	Description
ImageNet Dataset	We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. `imagenet/ ├── calibration_data │ ├── ILSVRC2012_val_00000001.JPEG │ ├── ... │ └── ILSVRC2012_val_00000100.JPEG ├── ILSVRC2017_val.txt ├── val │ ├── ILSVRC2012_val_00000001.JPEG │ ├── ... │ └── ILSVRC2012_val_00050000.JPEG └── val.txt`
COCO Dataset	We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. `coco/ ├── calibration_data │ ├── COCO_val2014_000000181007.jpg │ ├── ... │ └── COCO_val2014_000000181739.jpg └── coco_val2017 ├── annotations │ ├── instances_train2017.json │ └── instances_val2017.json └── images ├── 000000000139.jpg ├── 000000000285.jpg ├── ... ├── 000000581615.jpg └── 000000581781.jpg`
VOC Dataset	Attention Please note that the VOC2012 directory currently stores two datasets, VOC2007 and VOC2012, so please follow the directory structure below for the evaluation dataset. Please contact Horizon if you encounter any problems during data preparation. VOCdevkit/ └── VOC2012 ├── Annotations │ ├── 2007_000027.xml │ ├── ... │ └── 2012_004331.xml ├── ImageSets │ ├── Action │ │ ├── jumping_train.txt │ │ ├── jumping_trainval.txt │ │ ├── jumping_val.txt │ │ ├── ... │ │ ├── val.txt │ │ ├── walking_train.txt │ │ ├── walking_trainval.txt │ │ └── walking_val.txt │ ├── Layout │ │ ├── train.txt │ │ ├── trainval.txt │ │ └── val.txt │ ├── Main │ │ ├── aeroplane_train.txt │ │ ├── aeroplane_trainval.txt │ │ ├── aeroplane_val.txt │ │ ├── ... │ │ ├── train.txt │ │ ├── train_val.txt │ │ ├── trainval.txt │ │ ├── tvmonitor_train.txt │ │ ├── tvmonitor_trainval.txt │ │ ├── tvmonitor_val.txt │ │ └── val.txt │ └── Segmentation │ ├── train.txt │ ├── trainval.txt │ └── val.txt ├── JPEGImages │ ├── 2007_000027.jpg │ ├── ... │ └── 2012_004331.jpg ├── SegmentationClass │ ├── 2007_000032.png │ ├── ... │ └── 2011_003271.png ├── SegmentationObject │ ├── 2007_000032.png │ ├── ... │ └── 2011_003271.png └── train.txt
Cityscapes Dataset	We recommend that you unzip the downloaded dataset into the following structure. Please contact Horizon if you encounter any problems during data preparation. cityscapes/ ├── cityscapes_calibration_data │ ├── aachen_000000_000019_leftImg8bit.png │ ├── ... │ └── aachen_000099_000019_leftImg8bit.png ├── gtFine │ ├── test │ │ ├── berlin │ │ ├── ... │ │ └── munich │ ├── train │ │ ├── aachen │ │ ├── ... │ │ └── zurich │ └── val │ ├── frankfurt │ ├── lindau │ └── munster ├── leftImg8bit │ ├── test │ │ ├── berlin │ │ ├── ... │ │ └── munich │ ├── train │ │ ├── aachen │ │ ├── ... │ │ └── zurich │ └── val │ ├── frankfurt │ ├── lindau │ └── munster ├── license.txt └── README

Prepare Models

When using the model conversion sample package, please prepare the corresponding floating-point model first.

Note

If you need to do this process in the sample folder, you need to execute the 00_init.sh script in the folder first to get the corresponding original model and dataset.

Demo Of Algorithm Model Samples

Taking the Resnet50 model as an example, this section illustrates the steps of floating-point to fixed-point model conversion by using the scripts in classification/resnet50/.

Docker Container Prep

First complete the docker installation and configuration and enter the docker container accoridng to Docker Container Deployment .

Obtain The Original Model And Calibration Dataset

Execute 00_init.sh in the classification/resnet50/ folder to obtain the model and calibration dataset needed for the current sample.

# 1. Enter the folder the demo script resides
cd samples/onnx_ptq/classification/resnet50/
# 2. Execute the script to obtain the original model and calibration dataset
sh 00_init.sh

Check If the Model is Executable

Run the script as follows:

# Execute model checking
sh 01_check.sh

Prepare Calibration Dataset

Run the 02_preprocess.sh script in the same directory, as follows:

# Convert the images in common/calibration_data/imagenet
# to: ./calibration_data_rgb
sh 02_preprocess.sh

Note

We extracted 100 images from the ImageNet dataset and use them as a calibration dataset. Before calibration, we pre-processed the data and convert short size resize/crop size/NHWC to NCHW/to rgb.
The hb_compile tool will load data from the converted binary data, the format of the pre-processed binary data file is npy, the dtype is unit8.

Building Models

Run the 03_build.sh script in the same directory, as follows:

sh 03_build.sh

Note

The above script uses the hb_compile tool to convert the model. The most important thing to focus on is the conversion configuration file, refer to Model Quantized Compilation section.

The output of the above script is as follows:

ls model_output | cat
resnet50_224x224_nv12.hbm
resnet50_224x224_nv12_calibrated_model.onnx
resnet50_224x224_nv12_ptq_model.onnx
resnet50_224x224_nv12_optimized_float_model.onnx
resnet50_224x224_nv12_original_float_model.onnx
resnet50_224x224_nv12_quantized_model.bc
resnet50_224x224_nv12.html
resnet50_224x224_nv12.json 
resnet50_224x224_nv12_quant_info.json 
resnet50_224x224_nv12_advice.json
resnet50_224x224_nv12_node_info.csv
hb_compile.log

Note

For now you ONLY need to focus on the resnet50_224x224_nv12.hbm file.

Single Image Inference

Run the 04_inference.sh script to inference a single image, as follows:

sh 04_inference.sh

Note

As image pre-processing, model data post-processing are required in the image inference, we offered a Python sample script, please refer to sh 04_inference.sh.
This script is to perform the inference against a single image and verify whether the inference result meets the expectations. If you want to perform accuracy evaluation, refer to the script 05_evaluate.sh.

Accuracy Evaluation

Run the 05_evaluate.sh script to evaluate the accuracy, as follows:

export PARALLEL_PROCESS_NUM=${parallel_process_num}
sh 05_evaluate.sh

Note

As image pre-processing, model data post-processing are required in accuracy evaluation, awe offered a Python sample script, please refer to sh 05_evaluate.sh.
To accelerate the evaluation, you can adjust the number of concurrent processes by using the -p option; meanwhile, pay attention to the memory usage. When the value of the -p option is set to 0 or left unfilled, the fixed-point models in the CPU environment will be processed by 10 processes, while other scenarios using 1 process.

FAQ

Why is the reproduced accuracy slightly different from the indicators in the documentation?

There are two possible reasons:

There may be minor differences in calculation methods when in different server environments, which can cause small data fluctuations in the accuracy of the compiled fixed-point ONNX models in different server environments when compared with the documentation.
The different versions of third-party libraries such as OpenCV and NumPy, which may produce different results after pre-processing, and this may also lead to slight data fluctuations in reproduced accuracy when compared with the documentation.

No need to worry much about this situation, the records provided in the documentation is only for reference, and it is ok that your reproduced accuracy is slightly different from those in documents.

Why is the fixed-point model accuracy not aligned with the on-board accuracy of the hbm file in the ai_benchmark sample?

In the standard delivery, when adding the sample, we have already aligned the fixed-point model accuracy with the hbm on-board accuracy in the ai_benchmark sample.

If you find any unaligned accuracy, we recommend you first checking the model input for consistency.

When executing the fixed-point model evaluation script, you use the dataset of image type, while for the hbm model used on board, you use the binary dataset converted by the hb_eval_preprocess tool.

Based on this, if the dataset you used on-board is not generated by using the above methods, we recommend that you first use our data preprocessing tool (i.e., hb_eval_preprocess) to regenerate the dataset needed for on-board running on the same server that you run the fixed-point model accuracy and rerun the on-board accuracy to ensure the model inputs are consistent.

Attention

Make sure to use the same environment to generate the dataset by using the hb_eval_preprocess tool and to run the fixed-point model accuracy.

#Common Algorithm Model Samples

#Sample Location

#Prepare Datasets

#Dataset Download Address

#Dataset Reference Structure

#Prepare Models

#Demo Of Algorithm Model Samples

#Docker Container Prep

#Obtain The Original Model And Calibration Dataset

#Check If the Model is Executable

#Prepare Calibration Dataset

#Building Models

#Single Image Inference

#Accuracy Evaluation

#FAQ

#Why is the reproduced accuracy slightly different from the indicators in the documentation?

#Why is the fixed-point model accuracy not aligned with the on-board accuracy of the hbm file in the ai_benchmark sample?

Common Algorithm Model Samples

Sample Location

Prepare Datasets

Dataset Download Address

Dataset Reference Structure

Prepare Models

Demo Of Algorithm Model Samples

Docker Container Prep

Obtain The Original Model And Calibration Dataset

Check If the Model is Executable

Prepare Calibration Dataset

Building Models

Single Image Inference

Accuracy Evaluation

FAQ

Why is the reproduced accuracy slightly different from the indicators in the documentation?

Why is the fixed-point model accuracy not aligned with the on-board accuracy of the hbm file in the ai_benchmark sample?