#精度调优示例
本章以实际使用过程中遇到的精度问题为例，介绍 ONNX 模型的精度调优流程，请确保先看完精度调优指导章节，了解相关的理论知识和工具用法。
典型的精度问题包括：
- 全INT16量化精度达标，精度debug工具能够提供相对准确的敏感节点排序；
- 全INT16量化精度达标，设置大量敏感节点为高精度无法有效提升量化精度；
- 全INT16量化精度不达标，模型全BPU量化的前提下，进一步提高量化精度。
#敏感节点分析
精度Debug工具提供了节点量化敏感度的计算接口，能够计算各算子量化对输出结果的影响程度，将量化损失高的节点设置为高精度，完成精度调优。以HybridNets模型为例介绍该调优过程。
采用HMCT default INT8量化，校准算法选择percentile，校准精度未达标(det和ll_seg精度下降超1%)：
Model Float March Samples calibrated_model Cosine_Similarity
------------------------- ------- ------- --------- ------------------ -------------------
hybridnets-384-640_det 0.77222 nash-e 10000 0.75562(97.85%) 0.98012
hybridnets-384-640_da_seg 0.90467 nash-e 10000 0.89675(99.12%) 0.98012
hybridnets-384-640_ll_seg 0.85376 nash-e 10000 0.81813(95.83%) 0.98012#全INT16精度
首先设置all_node_type为INT16，校准算法选择percentile，此时校准精度满足要求，可以使用INT8+INT16混合精度完成调优：
quant_config = {"model_config": {"all_node_type": "int16"}}
Model Float March Samples calibrated_model Cosine_Similarity
------------------------- ------- ------- --------- ------------------ -------------------
hybridnets-384-640_det 0.77222 nash-e 10000 0.76866(99.54%) 0.997147
hybridnets-384-640_da_seg 0.90467 nash-e 10000 0.90405(99.93%) 0.997147
hybridnets-384-640_ll_seg 0.85376 nash-e 10000 0.84732(99.25%) 0.997147#混合精度调试
基于全INT16选择的percentile校准算法编译INT8校准模型，yaml文件中配置debug_mode为 "dump_calibration_data" 来保存校准数据，通过get_sensitivity_of_nodes输出节点量化敏感度：
hmct-debugger get-sensitivity-of-nodes hybridnets-384-640_calibrated_model.onnx calibration_data/ -n node -v True -s ./debug_result===========================node sensitivity============================
node cosine-similarity
-----------------------------------------------------------------------
/encoder/_blocks.0/_depthwise_conv/Conv 0.98768
/encoder/_swish/Mul 0.99526
/encoder/_blocks.2/_depthwise_conv/Conv 0.99852
/encoder/_blocks.0/Mul 0.99887
/encoder/_blocks.0/GlobalAveragePool 0.99889
/encoder/_blocks.2/_swish/Mul 0.99957
/bifpn/bifpn.5/conv3_up/depthwise_conv/conv/Conv 0.99964
/encoder/_blocks.0/_swish/Mul 0.99969
/encoder/_blocks.2/Mul 0.99979
/encoder/_blocks.2/GlobalAveragePool 0.9998
/bifpn/bifpn.2/conv3_up/pointwise_conv/conv/Conv 0.99983
/encoder/_blocks.2/_swish_1/Mul 0.99984
/bifpn/bifpn.5/p4_downsample/Pad 0.99985
...er/seg_blocks.4/block/block.0/block/block.0/Pad 0.99985
/encoder/_blocks.0/_project_conv/Conv 0.99986
/encoder/_blocks.5/_depthwise_conv/Conv 0.99988
/bifpn/bifpn.3/conv3_up/depthwise_conv/conv/Conv 0.99989
/classifier/conv_list.0/depthwise_conv/conv/Conv 0.99989
..._blocks.4/block/block.0/block/block.0/conv/Conv 0.99989
/regressor/conv_list.0/depthwise_conv/conv/Conv 0.99989
/bifpn/bifpn.2/conv3_up/depthwise_conv/conv/Conv 0.99992
/encoder/_blocks.17/_se_expand/Conv 0.99992
/encoder/_blocks.13/Mul 0.99992
/encoder/_blocks.1/_depthwise_conv/Conv 0.99992
/encoder/_blocks.13/GlobalAveragePool 0.99992
/encoder/_blocks.14/_se_expand/Conv 0.99992
/classifier/header/pointwise_conv/conv/Conv 0.99992
/encoder/_blocks.1/Add 0.99993
/encoder/_blocks.3/Mul 0.99993
/encoder/_blocks.3/GlobalAveragePool 0.99993
/encoder/_blocks.1/_swish/Mul 0.99993
/encoder/_blocks.15/Mul 0.99993
/encoder/_blocks.15/GlobalAveragePool 0.99993
/bifpn/bifpn.4/conv3_up/depthwise_conv/conv/Conv 0.99993
/bifpn/bifpn.1/conv3_up/pointwise_conv/conv/Conv 0.99993
/bifpn/bifpn.4/conv3_up/pointwise_conv/conv/Conv 0.99994
/encoder/_blocks.8/_project_conv/Conv 0.99994
/bifpn/bifpn.5/swish_3/Mul 0.99994
/bifpn/bifpn.3/conv3_up/pointwise_conv/conv/Conv 0.99994
/encoder/_blocks.8/GlobalAveragePool 0.99994
/encoder/_conv_stem/Conv 0.99994
/encoder/_blocks.13/_project_conv/Conv 0.99994
/encoder/_blocks.8/Mul 0.99994
/bifpn/bifpn.5/conv3_up/pointwise_conv/conv/Conv 0.99995
...按照余弦相似度排序从前往后的顺序，逐步设置算子INT16量化，校准模型精度也会随着增加，直到满足需求：
| 序号 | 余弦相似度阈值(<=该阈值设置为INT16) | 精度 | ||
|---|---|---|---|---|
| det | da_seg | ll_seg | ||
| 1 | None | 0.75562(97.85%) | 0.89675(99.12%) | 0.81813(95.83%) |
| 2 | 0.999 | 0.76531(99.11%) | 0.90274(99.79%) | 0.83874(98.24%) |
| 3 | 0.9998 | 0.76545(99.12%) | 0.90340(99.86%) | 0.83961(98.34%) |
| 4 | 0.9999 | 0.76613(99.21%) | 0.90420(99.95%) | 0.84216(98.64%) |
| 5 | 0.99992 | 0.76712(99.34%) | 0.90356(99.88%) | 0.84397(98.85%) |
| 6 | 0.99993 | 0.76781(99.43%) | 0.90374(99.90%) | 0.84484(98.95%) |
| 7 | 0.99994 | 0.76811(99.47%) | 0.90344(99.86%) | 0.84528(99.01%) |
由上述测试表格可得，将敏感度阈值小于等于0.99994的敏感节点设为INT16节点，校准精度满足需求：
quant_config = {
"model_config": {
"activation": {"calibration_type": "max", "max_percentile": 0.99995},
},
"node_config": {
"/encoder/_blocks.0/_depthwise_conv/Conv": {"qtype": "int16"},
"/encoder/_swish/Mul": {"qtype": "int16"},
"/encoder/_blocks.2/_depthwise_conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.0/Mul": {"qtype": "int16"},
"/encoder/_blocks.0/GlobalAveragePool": {"qtype": "int16"},
"/encoder/_blocks.2/_swish/Mul": {"qtype": "int16"},
"/bifpn/bifpn.5/conv3_up/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.0/_swish/Mul": {"qtype": "int16"},
"/encoder/_blocks.2/Mul": {"qtype": "int16"},
"/encoder/_blocks.2/GlobalAveragePool": {"qtype": "int16"},
"/bifpn/bifpn.2/conv3_up/pointwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.2/_swish_1/Mul": {"qtype": "int16"},
"/bifpn/bifpn.5/p4_downsample/Pad": {"qtype": "int16"},
"/bifpndecoder/seg_blocks.4/block/block.0/block/block.0/Pad": {"qtype": "int16"},
"/encoder/_blocks.0/_project_conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.5/_depthwise_conv/Conv": {"qtype": "int16"},
"/bifpn/bifpn.3/conv3_up/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/classifier/conv_list.0/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/bifpndecoder/seg_blocks.4/block/block.0/block/block.0/conv/Conv": {"qtype": "int16"},
"/regressor/conv_list.0/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/bifpn/bifpn.2/conv3_up/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.17/_se_expand/Conv": {"qtype": "int16"},
"/encoder/_blocks.13/Mul": {"qtype": "int16"},
"/encoder/_blocks.1/_depthwise_conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.13/GlobalAveragePool": {"qtype": "int16"},
"/encoder/_blocks.14/_se_expand/Conv": {"qtype": "int16"},
"/classifier/header/pointwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.1/Add": {"qtype": "int16"},
"/encoder/_blocks.3/Mul": {"qtype": "int16"},
"/encoder/_blocks.3/GlobalAveragePool": {"qtype": "int16"},
"/encoder/_blocks.1/_swish/Mul": {"qtype": "int16"},
"/encoder/_blocks.15/Mul": {"qtype": "int16"},
"/encoder/_blocks.15/GlobalAveragePool": {"qtype": "int16"},
"/bifpn/bifpn.4/conv3_up/depthwise_conv/conv/Conv": {"qtype": "int16"},
"/bifpn/bifpn.1/conv3_up/pointwise_conv/conv/Conv": {"qtype": "int16"},
"/bifpn/bifpn.4/conv3_up/pointwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.8/_project_conv/Conv": {"qtype": "int16"},
"/bifpn/bifpn.5/swish_3/Mul": {"qtype": "int16"},
"/bifpn/bifpn.3/conv3_up/pointwise_conv/conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.8/GlobalAveragePool": {"qtype": "int16"},
"/encoder/_conv_stem/Conv": {"qtype": "int16"},
"/encoder/_blocks.13/_project_conv/Conv": {"qtype": "int16"},
"/encoder/_blocks.8/Mul": {"qtype": "int16"},
},
}
Model Float March Samples calibrated_model Cosine_Similarity
------------------------- ------- ------- --------- -------------------- -----------------
hybridnets-384-640_det 0.77222 nash-e 10000 0.76811(99.47%) 0.994576
hybridnets-384-640_da_seg 0.90467 nash-e 10000 0.90344(99.86%) 0.994576
hybridnets-384-640_ll_seg 0.85376 nash-e 10000 0.84528(99.01%) 0.994576完整的精度调优部署示例见：HybridNets精度调优部署示例。
#敏感节点失效
当使用精度Debug工具设置敏感节点为高精度无法有效提升模型精度时，可先尝试指定输出节点，过滤掉不相关的节点，此外观察模型输出误差选择其他评估指标，提高敏感度排序和精度的相关性，更进一步通过分析模型结构，将量化损失风险较大的典型子结构(模型输出、输入以及具有特定物理意义的结构等)设为高精度，完成精度调优。以YoloP模型为例介绍该调优过程。
采用HMCT default INT8量化，校准算法选择percentile，校准精度未达标(det精度下降超1%)：
Model Float March Samples calibrated_model Cosine_Similarity
-------------------- ------- ------- --------- ------------------ -------------------
yolop-384-640_det 0.76448 nash-e 10000 0.61507(80.46%) 0.999891
yolop-384-640_da_seg 0.89008 nash-e 10000 0.88863(99.84%) 0.999891
yolop-384-640_ll_seg 0.6523 nash-e 10000 0.65357(100.19%) 0.999891#全INT16精度
首先设置all_node_type为INT16，校准算法选择percentile，此时校准精度满足要求，可以使用INT8+INT16混合精度完成调优：
quant_config = {"model_config": {"all_node_type": "int16"}}
Model Float March Samples calibrated_model Cosine_Similarity
-------------------- ------- ------- --------- ------------------ -----------------
yolop-384-640_det 0.76448 nash-e 10000 0.75890(99.27%) 0.99999
yolop-384-640_da_seg 0.89008 nash-e 10000 0.88950(99.93%) 0.99999
yolop-384-640_ll_seg 0.6523 nash-e 10000 0.64821(99.37%) 0.99999 #混合精度调试
基于全INT16选择的percentile校准算法编译INT8校准模型，yaml文件中配置debug_mode为"dump_calibration_data"来保存校准数据，通过get_sensitivity_of_nodes输出节点量化敏感度：
hmct-debugger get-sensitivity-of-nodes yolop-384-640_calibrated_model.onnx calibration_data/ -n node -v True -s ./debug_result=======================node sensitivity========================
node cosine-similarity
---------------------------------------------------------------
Mul_943 0.99736
Mul_647 0.99894
Mul_795 0.99909
Conv_50 0.99976
Div_49 0.99983
Conv_92 0.99989
Div_58 0.9999
Conv_1119 0.99995
Conv_88 0.99996
Conv_41 0.99996
Conv_59 0.99998
Slice_4 0.99998
Slice_9 0.99998
Slice_14 0.99998
Slice_19 0.99998
Slice_24 0.99998
Slice_29 0.99998
Slice_34 0.99998
Slice_39 0.99998
Concat_40 0.99998
Div_67 0.99998
MaxPool_297 0.99999
MaxPool_298 0.99999
MaxPool_299 0.99999
Concat_300 0.99999
Concat_1003 0.99999
Conv_177 0.99999
Div_296 0.99999
ScatterND_705 0.99999
Slice_645 0.99999
Reshape_706 0.99999
Conv_110 0.99999
Conv_87 0.99999
Concat_89 0.99999
LeakyRelu_91 0.99999
Mul_584 0.99999
ScatterND_640 0.99999
Concat_1105 0.99999
LeakyRelu_1107 0.99999
Conv_1004 0.99999
Add_582 0.99999
Conv_119 0.99999
Resize_1014 0.99999
Conv_1015 0.99999
Conv_1043 0.99999
Conv_266 0.99999
Conv_199 0.99999
Div_100 0.99999
...按照余弦相似度排序从前往后顺序，逐步设置算子INT16量化，然而即使将大量敏感节点设为INT16，精度也未能达标：
| 序号 | 余弦相似度阈值(<=该阈值设置为INT16) | 精度 | ||
|---|---|---|---|---|
| det | da_seg | ll_seg | ||
| 1 | None | 0.61507(80.46%) | 0.88863(99.84%) | 0.65357(100.19%) |
| 2 | 0.9999 | 0.60956(79.74%) | 0.88911(99.89%) | 0.65925(101.07%) |
| 3 | 0.99996 | 0.60978(79.76%) | 0.88933(99.92%) | 0.66112(101.35%) |
| 4 | 0.99998 | 0.60956(79.73%) | 0.88931(99.91%) | 0.66125(101.37%) |
| 5 | 0.99999 | 0.66426(86.89%) | 0.88958(99.94%) | 0.66065(101.28%) |
观察INT8校准模型的精度结果，只有det分支的精度未达到99%，通过get_sensitivity_of_nodes接口计算节点量化敏感度时，可以通过-o选项指定det输出对应的节点，仅计算影响det输出的敏感度排序，提高精度问题定位准确度：
hmct-debugger get-sensitivity-of-nodes yolop-384-640_calibrated_model.onnx calibration_data/ -n node -o Concat_1003 -v True -s det_debug_result/=======================node sensitivity========================
node cosine-similarity
---------------------------------------------------------------
Mul_943 0.99736
Mul_647 0.99894
Mul_795 0.99909
Conv_50 0.99997
Div_58 0.99997
Div_49 0.99999
Concat_1003 0.99999
ScatterND_705 0.99999
Slice_645 0.99999
Reshape_706 0.99999
Mul_584 0.99999
ScatterND_640 0.99999
Add_582 0.99999
Conv_41 0.99999
Slice_4 0.99999
Slice_9 0.99999
Slice_14 0.99999
Slice_19 0.99999
Slice_24 0.99999
Slice_29 0.99999
Slice_34 0.99999
Slice_39 0.99999
Concat_40 0.99999
Conv_88 0.99999
Conv_59 0.99999
...按照余弦相似度排序从前往后顺序，设置算子INT16量化，仅关注det输出能过滤掉无用节点，但最终精度仍未达标：
| 序号 | 余弦相似度阈值(<=该阈值设置为INT16) | 精度 | ||
|---|---|---|---|---|
| det | da_seg | ll_seg | ||
| 1 | None | 0.61507(80.46%) | 0.88863(99.84%) | 0.65357(100.19%) |
| 2 | 0.9999 | 0.60868(79.62%) | 0.88836(99.81%) | 0.65300(100.11%) |
| 3 | 0.99997 | 0.60961(79.74%) | 0.88902(99.88%) | 0.65664(100.66%) |
| 4 | 0.99999 | 0.66461(86.94%) | 0.88932(99.91%) | 0.65876(100.99%) |
观察INT8校准模型的输出相似度，其中det分支的L1和L2距离同浮点偏差很大，尝试将余弦相似度替换为其他指标：
hmct-info yolop-384-640_calibrated_model.onnx -c ./calibration_data/images/00.npy
INFO:root:The quantized model output:
=================================================================================
Output Cosine Similarity L1 Distance L2 Distance Chebyshev Distance
---------------------------------------------------------------------------------
det_out 0.995352 7.633481 289.637665 552.817566
drive_area_seg 0.998973 0.004005 0.001132 0.592610
lane_line_seg 0.999933 0.000417 0.000069 0.564768通过get_sensitivity_of_nodes接口计算节点敏感度时，可以指定mse作为评估指标，提高不同节点之间的区分度：
hmct-debugger get-sensitivity-of-nodes yolop-384-640_calibrated_model.onnx calibration_data/ -n node -o Concat_1003 -m mse -v True -s det_mse_debug_result/===================node sensitivity====================
node mse
-------------------------------------------------------
Mul_943 164.82712
Mul_647 65.88637
Mul_795 56.86866
Conv_50 2.04226
Div_58 1.88065
Concat_1003 0.87797
Div_49 0.84962
ScatterND_705 0.67858
Slice_645 0.67379
Reshape_706 0.67379
ScatterND_640 0.55187
Mul_584 0.54884
Add_582 0.52263
Conv_41 0.4714
Slice_4 0.38413
Slice_9 0.38413
Slice_14 0.38413
Slice_19 0.38413
Slice_24 0.38413
Slice_29 0.38413
Slice_34 0.38413
Slice_39 0.38413
Concat_40 0.38413
Conv_88 0.35534
Conv_59 0.33164
Conv_92 0.30398
Div_67 0.16826
ScatterND_853 0.16711
Slice_793 0.1627
Reshape_854 0.1627
ScatterND_788 0.13494
Mul_732 0.132
Add_730 0.12381
ScatterND_1001 0.07478
Conv_550 0.07226
Conv_518 0.06468
Conv_546 0.06344
Conv_310 0.05974
Conv_68 0.05735
Conv_338 0.05684
Add_86 0.04319
ScatterND_936 0.0428
Concat_89 0.04201
LeakyRelu_91 0.04201
Concat_517 0.04193
Conv_87 0.04148
Slice_941 0.04148
Reshape_1002 0.04148
Conv_448 0.04034
MaxPool_297 0.03739
MaxPool_298 0.03739
MaxPool_299 0.03739
Concat_300 0.03739
Mul_880 0.03334
Div_100 0.03328
Conv_855 0.03146
Concat_547 0.03094
LeakyRelu_549 0.03094
Add_878 0.03016
Div_558 0.02966
...按照mse相似度排序从前往后顺序，设置算子高精度，最终在增加大量INT16节点后精度才能够达标：
| 序号 | MSE阈值(>=该阈值设置为INT16) | 精度 | ||
|---|---|---|---|---|
| det | da_seg | ll_seg | ||
| 1 | None | 0.61507(80.46%) | 0.88863(99.84%) | 0.65357(100.19%) |
| 2 | 0.5 | 0.66471(86.95%) | 0.88902(99.88%) | 0.65664(100.66%) |
| 3 | 0.2 | 0.66447(86.92%) | 0.88933(99.92%) | 0.66112(101.35%) |
| 4 | 0.1 | 0.72969(95.45%) | 0.88931(99.91%) | 0.66125(101.37%) |
| 5 | 0.05 | 0.73393(96.00%) | 0.88934(99.92%) | 0.66121(101.37%) |
| 6 | 0.04 | 0.73321(95.91%) | 0.88931(99.91%) | 0.66125(101.37%) |
| 7 | 0.03 | 0.75707(99.03%) | 0.88944(99.93%) | 0.66137(101.39%) |
#分析子图结构
即使是基于mse指标只关注det输出的提升，设置大量敏感节点高精度仍无法有效提升精度，更进一步考虑到当前仅det任务不达标，推断da_seg分支、ll_seg分支以及公共的backbone部分存在敏感节点的可能性较小，关注det分支的精度调优，结合模型结构分析，尝试指定det输出位置子图采用高精度量化，测试精度：
quant_config = {
"subgraph_config": {
"det_head": {
"inputs": ["Conv_559", "Conv_707", "Conv_855"],
"outputs": ["Concat_1003"],
"qtype": "int16",
},
}
}
Model Float March Samples calibrated_model Cosine_Similarity
-------------------- ------- ------- --------- ------------------ -------------------
yolop-384-640_det 0.76448 nash-e 10000 0.76275(99.77%) 0.99991
yolop-384-640_da_seg 0.89008 nash-e 10000 0.88863(99.84%) 0.99991
yolop-384-640_ll_seg 0.6523 nash-e 10000 0.65357(100.19%) 0.99991注意
当敏感节点排序不准确时，通过配置子图高精度，初步确定损失来源，若子图高精度耗时增加显著，可以子图内进行敏感度分析，减少高精度算子占比。
完整的精度调优部署示例见：YoloP精度调优部署示例。
#精度调优技巧
整体调优流程需不断修改节点的高精度配置，编译生成模型，进行精度验证，但完整的编译链路耗时长，调试成本高。基于此，我们提供了IR接口支持您直接对calibrated_model.onnx模型的量化参数进行修改，从而快速验证。
from hmct.ir import load_model, save_model
from hmct.common import find_input_calibration, find_output_calibration
model = load_model("calibrated_model.onnx")
# 修改特定激活/权重校准节点 采用特定的数据类型
node = model.graph.node_mappings["ReduceMax_1317_HzCalibration"]
print(node.qtype) # 支持读取node的数据类型
node.qtype = "float32" # 支持配置int8,int16,float16,float32
# 配置所有激活/权重校准节点 采用int16量化
calibration_nodes = model.graph.type2nodes["HzCalibration"]
# 配置所有激活节点采用int16
for node in calibration_nodes:
if node.tensor_type == "feature":
node.qtype = "int16"
# 配置所有权重节点采用int16
for node in calibration_nodes:
if node.tensor_type == "weight":
node.qtype = "int16"
# 配置所有校准节点采用int16
for node in calibration_nodes:
node.qtype = "int16"
# 配置某一个普通节点采用int16
for node in model.graph.nodes:
if node.name in ["Conv_0"]:
for i in range(len(node.inputs)):
input_calib = find_input_calibration(node, i)
# 要求能够在输入找到HzCalibration,并且tensor_type为feature类型
if input_calib and input_calib.tensor_type == "feature":
input_calib.qtype = "int16"
# 配置某一个普通节点输出为int16
for node in model.graph.nodes:
if node.name in ["Conv_0"]:
output_calib = find_output_calibration(node)
# 要求能够在输出找到HzCalibration,并且tensor_type为feature类型
if output_calib and output_calib.tensor_type == "feature":
input_calib.qtype = "int16"
# 配置某个节点类型采用int16
for node in model.graph.nodes:
if node.op_type in ["Conv"]:
for i in range(len(node.inputs)):
input_calib = find_input_calibration(node, i)
# 要求能够在输入找到HzCalibration,并且tensor_type为feature类型
if input_calib and input_calib.tensor_type == "feature":
input_calib.qtype = "int16"
# 修改特定激活/权重校准节点 采用特定的阈值
node = model.graph.node_mappings["ReduceMax_1317_HzCalibration"]
print(node.thresholds) # 支持读取node的阈值结果
node.thresholds = [4.23] # 支持np.array, List[float]
save_model(model, "calibrated_model_modified.onnx")