This chapter introduces the post-training quantization(PTQ) accuracy tuning pipeline using the precision problems encountered in actual use as an example. Please read the Model Accuracy Tuning chapter firstly to Understand relevant theoretical knowledge and tool usage.
Typical accuracy issues include:
The all node type int16 quantization accuracy meets the requirements and Accuracy Debug Tool can provide relatively accurate sorting of sensitive nodes;
The all node type int16 quantization accuracy meets the requirements, but setting a large number of sensitive nodes to higher precision cannot effectively improve quantization accuracy;
The all node type int16 quantization accuracy does not meet the standard, under the premise of full BPU quantization of the model, we hope to further improve the quantization accuracy.
Accuracy Debug Tool provides an interface for calculating node quantization sensitivity. It can calculate the impact of each operator's quantization on the output results, set nodes with high quantization loss to higher precision, and complete accuracy tuning. The tuning pipeline is described using the HybridNets model as an example.
Using HMCT default INT8 quantization, percentile is selected as the calibration algorithm, the calibration accuracy does not meet the requirements (the accuracy of det and ll_seg decreases by more than 1%):
First, set all_node_type to INT16 and select percentile for the calibration algorithm. At this point, the calibration accuracy meets the requirements, and we can use INT8+INT16 mixed precision to complete the tuning:
Compile the INT8 calibration model based on the percentile calibration algorithm selected for INT16 model, configure debug_mode: "dump_calibration_data" in the yaml file to save the calibration data, and output the node quantization sensitivity through get_sensitivity_of_nodes:
Sorting by cosine similarity from front to back, gradually set the operator INT16 quantization, and the calibration model accuracy will increase until it meets the requirements:
Serial Number
Cosine Similarity Value(<=value will be set to INT16)
Accuracy
det
da_seg
ll_seg
1
None
0.75562(97.85%)
0.89675(99.12%)
0.81813(95.83%)
2
0.999
0.76531(99.11%)
0.90274(99.79%)
0.83874(98.24%)
3
0.9998
0.76545(99.12%)
0.90340(99.86%)
0.83961(98.34%)
4
0.9999
0.76613(99.21%)
0.90420(99.95%)
0.84216(98.64%)
5
0.99992
0.76712(99.34%)
0.90356(99.88%)
0.84397(98.85%)
6
0.99993
0.76781(99.43%)
0.90374(99.90%)
0.84484(98.95%)
7
0.99994
0.76811(99.47%)
0.90344(99.86%)
0.84528(99.01%)
From the above test table, we can see that if the sensitive nodes with a sensitivity value less than or equal to 0.99994 are set as INT16 nodes, the calibration accuracy meets the requirements:
If using the Accuracy Debug Tool to set sensitive nodes to higher precision fails to effectively improve model accuracy, we can first try specifying output nodes to filter out irrelevant nodes. Additionally, observe the model output error and select other metric to improve the correlation between sensitivity ranking and precision. Furthermore, by analyzing the model structure, typical substructures with a higher risk of quantization loss (such as model outputs, inputs, and structures with specific physical meaning) can be set to higher precision to complete accuracy tuning. This tuning pipeline is described using the YoloP model as an example.
Using HMCT default INT8 quantization, percentile is selected as the calibration algorithm, the calibration accuracy does not meet the requirements (the accuracy of det decreases by more than 1%):
First, set all_node_type to INT16 and select percentile for the calibration algorithm. At this point, the calibration accuracy meets the requirements, and we can use INT8+INT16 mixed precision to complete the tuning:
Compile the INT8 calibration model based on the percentile calibration algorithm selected for INT16 model, configure debug_mode: "dump_calibration_data" in the yaml file to save the calibration data, and output the node quantization sensitivity through get_sensitivity_of_nodes:
Sorting by cosine similarity from front to back, gradually set the operator INT16 quantization. However, even with a large number of sensitive nodes set to INT16, the accuracy still failed to meet the requirements:
Serial Number
Cosine Similarity Value(<=value will be set to INT16)
Accuracy
det
da_seg
ll_seg
1
None
0.61507(80.46%)
0.88863(99.84%)
0.65357(100.19%)
2
0.9999
0.60956(79.74%)
0.88911(99.89%)
0.65925(101.07%)
3
0.99996
0.60978(79.76%)
0.88933(99.92%)
0.66112(101.35%)
4
0.99998
0.60956(79.73%)
0.88931(99.91%)
0.66125(101.37%)
5
0.99999
0.66426(86.89%)
0.88958(99.94%)
0.66065(101.28%)
Observing the accuracy results of the INT8 calibrated model, only det branch does not reach 99%. When calculating nodes sensitivity through get_sensitivity_of_nodes interface, we can use the -o option to specify det output, so that a better sensitivity ranking can be obtained:
Sorting by cosine similarity from front to back, setting the operator to INT16 quantization, and focusing only on the det output can filter out useless nodes, but the final accuracy still does not meet the requirements:
Serial Number
Cosine Similarity Value(<=value will be set to INT16)
Accuracy
det
da_seg
ll_seg
1
None
0.61507(80.46%)
0.88863(99.84%)
0.65357(100.19%)
2
0.9999
0.60868(79.62%)
0.88836(99.81%)
0.65300(100.11%)
3
0.99997
0.60961(79.74%)
0.88902(99.88%)
0.65664(100.66%)
4
0.99999
0.66461(86.94%)
0.88932(99.91%)
0.65876(100.99%)
Observe the output similarity of the INT8 calibrated model. The L1 and L2 distances of the det branch deviate greatly from the floating point. Try replacing cosine similarity with other metrics:
When calculating nodes sensitivity through the get_sensitivity_of_nodes interface, we can specify mse as the metric to improve the discrimination between different nodes:
Sort by mse similarity from front to back, set the operator to higher precision, and finally achieve the required accuracy after adding a large number of INT16 nodes:
Even if we only focus on improving the det output based on mse metric, setting a large number of sensitive nodes for higher precision still cannot effectively improve the accuracy. Furthermore, considering that only the det task currently does not meet the requirements, it is inferred that quantization loss is less likely to occur in nodes in the da_seg and ll_seg branches, as well as in the shared backbone. Focus on the accuracy tuning of det branch, combined with model structure analysis, try to specify the det output position subgraph to use higher precision and test the calibration accuracy:
When the sorting of sensitive nodes is inaccurate, the source of the loss can be preliminarily determined by configuring the subgraph for higher precision. If the latency of the subgraph with higher precision increases significantly, sensitivity analysis can be performed within the subgraph, so that fewer nodes with higher precision will be configured.
Due to hardware constraints and latency, when set all_node_type to INT16, nodes with int8 precision still exist in model, including weights of Conv and ConvTranspose, Resize, GridSample and the second input of MatMul. PTQ introduces an identical operator to compensate for the quantization loss caused by int8 precision, further improving the model's accuracy with all nodes deployed on BPU, and complete accuracy tuning. The Lane model is used as an example to illustrate the tuning pipeline.
Using HMCT default INT8 quantization, max calibration algorithm with asymmetric and per-channel is selected, the accuracy does not meet the requirements(average cosine similarity of all outputs is less than 0.99):
First, set all_node_type to INT16 and select max for the calibration algorithm. At this point, the calibration accuracy meets the requirements(average cosine similarity of occlusion and box_arrow outputs does not reach 0.99), further accuracy tuning is required:
Since INT8 precision nodes still exist in model after setting all_node_type to INT16, we can modify the qtype of all calibration nodes to INT16 through IR interface provided by HMCT to obtain a true INT16 model:
from hmct.ir import load_model, save_modelmodel = load_model("lane_calibrated_model_int16.onnx")calibration_nodes = model.graph.type2nodes["HzCalibration"]for node in calibration_nodes: node.qtype = "int16"save_model(model, "lane_calibrated_model_real_int16.onnx")
Verify that the average similarity of the true INT16 calibrated model on all outputs, which can meet requirements, so that accuracy can be improved by compensating quantization error.
The analysis process of compensation needs to be based on the calibrated model configured with all_node_type int16, nodes sensitivity is output through get_sensitivity_of_nodes:
Then modify the calibrated model according to the nodes sensitivity sorting, and increase the quantization precision from INT8 to INT16 until occlusion and box_arrow meet requirements:
from hmct.common import find_input_calibration, find_output_calibrationfrom hmct.ir import load_model, save_modelmodel = load_model("lane_calibrated_model_int16.onnx")improved_nodes = ["Conv_360", "Conv_3", "Conv_338"]for node in model.graph.nodes: if node.name not in improved_nodes: continue if node.op_type in ["Conv", "ConvTranspose", "MatMul"]: input1_calib = find_input_calibration(node, 1) if input1_calib and input1_calib.tensor_type == "weight": input1_calib.qtype = "int16" if node.op_type == "Resize": input_calib = find_input_calibration(node, 0) if input_calib and input_calib.tensor_type == "feature": input_calib.qtype = "int16" interpolation_mode = node.attributes.get("mode", "nearest") # In nearest mode, output quantization precision can be improved # to be close to int16, in other modes, only the input quantization # precision can be improved to be close to int16. if interpolation_mode == "nearest": output_calib = find_output_calibration(node) if output_calib and output_calib.tensor_type == "feature": output_calib.qtype = "int16" if node.op_type == "GridSample": input_calib = find_input_calibration(node, 0) if input_calib and input_calib.tensor_type == "feature": input_calib.qtype = "int16" interpolation_mode = node.attributes.get("mode", "bilinear") # In nearest mode, output quantization precision can be improved # to be close to int16, in other modes, only the input quantization # precision can be improved to be close to int16. if interpolation_mode == "nearest": output_calib = find_output_calibration(node) if output_calib and output_calib.tensor_type == "feature": output_calib.qtype = "int16"save_model(model, "lane_calibrated_model_int16_improved.onnx")
Serial Number
Cosine Similarity Value(<=value will be set to INT16)
Output Cosine
occlusion
box_arrow
Min
Avg
Min
Avg
1
None
0.943404
0.983970
0.755858
0.987496
2
0.999
0.983739
0.997729
0.893116
0.994958
3
0.99
0.952758
0.994116
0.745781
0.987434
As shown in the table above, increasing the weight quantization precision of Conv_360, Conv_3, and Conv_338 from INT8 to INT16 allows all output similarities to meet requirements. To deploy all nodes on the BPU, a similar operator is introduced to compensate for the quantization loss caused by int8 precision, and the final precision will be improved to close to INT16, for details regarding ec, please refer to The quant_config Introduction section.
It is recommended that Resize and GridSample use the nearest sampling mode. In this case, the operator's output will not introduce any new values, and error can be compensated. Otherwise, INT8 quantization of the new value on output would introduce additional loss which cannot be compensated.
After compensating for the weight quantization losses of Conv_360, Conv_3, and Conv_338, the accuracy of calibrated model with all_node_type int16 configured can meet requirements. Then attempted to optimize the INT8 calibrated model with error compensated. The accuracy of the INT8 calibrated model is as follows:
Sorting by cosine similarity from front to back, gradually setting the operator INT16 quantization, the calibrated model similarity will also increase:
Serial Number
Cosine Similarity Value
Output Cosine
mask
field
attr
backgroud
cls
box
cls_sl
box_sl
occlusion
cls_arrow
box_arrow
1
None
0.874048
0.875366
0.908879
0.976282
0.959749
0.939461
0.959271
0.947069
0.945837
0.942194
0.829580
2
0.99
0.980708
0.987483
0.989023
0.993368
0.991154
0.985205
0.990900
0.990375
0.985721
0.975350
0.963180
3
0.999
0.988858
0.990837
0.994669
0.995994
0.995570
0.994660
0.995466
0.996202
0.991201
0.979100
0.980218
4
0.9995
0.991085
0.991593
0.995369
0.997524
0.995818
0.995149
0.996001
0.996851
0.992633
0.981471
0.982875
Based on the above table, we optimized model by setting sensitive nodes with a sensitivity value less than or equal to 0.9995 to INT16. Except for cls_arrow and box_arrow, average similarity of all other outputs was no less than 0.99. Observing that cls_arrow and box_arrow share the same branch, we additionally tried configuring arrow output head subgraph to INT16, quantization configuration and output similarity are as follows:
Sort by cosine similarity from front to back, and gradually set the operator INT16 quantization until box_arrow meets the requirements:
Serial Number
Cosine Similarity Value
Output Cosine
mask
filed
attr
backgroud
cls
box
cls_sl
box_sl
occlusion
cls_arrow
box_arrow
1
None
0.991085
0.991593
0.995369
0.997524
0.995818
0.995149
0.996001
0.996851
0.992633
0.990650
0.983423
2
0.999
0.993978
0.993429
0.996853
0.998160
0.996915
0.996035
0.997124
0.997249
0.994457
0.992993
0.989119
3
0.9995
0.995126
0.994426
0.997494
0.998554
0.997245
0.996941
0.997775
0.998297
0.995518
0.995444
0.990272
Finally, by setting some sensitive nodes to INT16, the average similarities of all model outputs meet the requirements. The quantization configuration and output similarity are as follows:
The PTQ accuracy tuning pipeline requires constant modification of node configurations, model compilation and accuracy verification. The entire process is time-consuming and expensive to debug. Based on this, we provide the IR interface to support you to directly modify quantization parameters in calibrated_model.onnx for rapid verification.
from hmct.ir import load_model, save_modelfrom hmct.common import find_input_calibration, find_output_calibrationmodel = load_model("calibrated_model.onnx")# Modify specific activation or weight calibration nodes to use specific qtypenode = model.graph.node_mappings["ReduceMax_1317_HzCalibration"]print(node.qtype) # Support qtype readingnode.qtype = "float32" # Support int8, int16, float16, float32 configured# Configure all activation or weight calibration nodes to use int16 qtypecalibration_nodes = model.graph.type2nodes["HzCalibration"]# Configure all activation calibration nodes to use int16 qtypefor node in calibration_nodes: if node.tensor_type == "feature": node.qtype = "int16"# Configure all weight calibration nodes to use int16 qtypefor node in calibration_nodes: if node.tensor_type == "weight": node.qtype = "int16"# Configure all calibration nodes to use int16 qtypefor node in calibration_nodes: node.qtype = "int16"# Configure a node with int16 input qtypefor node in model.graph.nodes: if node.name in ["Conv_0"]: for i in range(len(node.inputs)): input_calib = find_input_calibration(node, i) # It is required to be able to find HzCalibration in the # input, and tensor_type is feature. if input_calib and input_calib.tensor_type == "feature": input_calib.qtype = "int16"# Configure a node with int16 output qtypefor node in model.graph.nodes: if node.name in ["Conv_0"]: output_calib = find_output_calibration(node) # It is required to be able to find HzCalibration in the # input, and tensor_type is feature. if output_calib and output_calib.tensor_type == "feature": input_calib.qtype = "int16"# Configure nodes with specific op_type to int16for node in model.graph.nodes: if node.op_type in ["Conv"]: for i in range(len(node.inputs)): input_calib = find_input_calibration(node, i) # It is required to be able to find HzCalibration in the # input, and tensor_type is feature. if input_calib and input_calib.tensor_type == "feature": input_calib.qtype = "int16"# Modify specific activation or weight calibration nodes with specific thresholdsnode = model.graph.node_mappings["ReduceMax_1317_HzCalibration"]print(node.thresholds) # Support thresholds readingnode.thresholds = [4.23] # Support np.array, List[float]save_model(model, "calibrated_model_modified.onnx")
