• English
  • 3.9.0

    • Deployment Consistency Analysis

    When you use the PTQ method for ONNX model quantization compilation, the following outputs will be generated in phases (where xxx represents the name of your model):

    File NameDescription
    xxx_original_float_model.onnx

    Original floating-point model.
    Perform infer shape on your model and conduct a legitimacy check on operators and tensors, which serves as the baseline model for subsequent processing.

    xxx_optimized_float_model.onnx

    Optimized floating-point model.
    Enhance the on-board execution performance of the model through a series of equivalent model optimizations, including but not limited to the following operations:

    • Constant Folding
    • Operator Replacement
    • Operator Fusion
    • Operator Elimination
    • Operator Reordering

    xxx_calibrated_model.onnx

    Calibrated (fake-quantized) model.
    According to the quant_config you configured, perform Fake Quantization on the model to simulate the quantization behavior under different bit widths in the floating-point domain.

    xxx_ptq_model.onnx

    PTQ model.
    Perform fixed-point processing on operators such as lookup table (LUT) type ones to further approximate the real fixed-point execution behavior.

    xxx_quantized_model.bc

    Fixed-point model (in bc format format).
    Convert the ONNX model to the HBIR model (in bc format format), and complete full-model fixed-point processing.

    xxx.hbm

    On-board executable model.
    Compile the HBIR model into an instruction set file that can be directly loaded and run on the board.

    Model Consistency and Accuracy Troubleshooting

    If you observe a significant accuracy degradation during deployment, or suspect consistency issues between different model stages, it is recommended to use the The hb_verifier Tool for analysis.

    The hb_verifier tool supports node-by-node output comparison between any two model stages using cosine similarity, helping to identify where discrepancies are introduced.

    Attention

    The hbm model only supports consistency comparison of the final outputs and does not allow per-node analysis.

    The reference command is as follows:

    hb_verifier \
  -m ./xxx_original_float_model.onnx,./xxx_quantized_model.bc \
  -i ./input.npy

    Case 1: All operator-level and final output cosine similarities are normal

    • No abnormal cosine similarity is observed at any operator or at the final output

    • Recommended actions:

      1. Prioritize checking whether the input data preprocessing pipeline of the accuracy verification process (such as normalization, color space, dimension order, etc.) is consistent with that in the training phase.

      2. If it is confirmed that the data processing is correct, please contact the Horizon Robotics technical support personnel for problem analysis.

    Case 2: All operator-level cosine similarities are abnormal

    • Usually caused by input data anomalies or format mismatches.

    • Recommended handling methods:

      1. Check the source, format and preprocessing logic of the input data.

      2. If the input data is confirmed to be correct, please contact Horizon Robotics technical support personnel for problem analysis.

    Case 3: Low cosine similarity is observed on some operators

    • Indicates that the error may be introduced at specific model stages or specific operators.

    • Recommended handling methods: Based on the intermediate outputs, use hb_verifier to perform phased consistency comparison and analysis, for specific analysis methods, refer to the detailed instructions on accuracy and consistency analysis of the model at different stages below.

    Expected Accuracy and Consistency Across Model Stages

    1. xxx_original_float_model.onnx to xxx_optimized_float_model.onnx

      • This stage only includes equivalent model optimization.

      • Expected results: The output cosine similarity should be ≥ 99.9%, and the model accuracy performance should remain consistent.

      • If a decrease in accuracy is observed, please contact Horizon Robotics technical support personnel for problem analysis.

    1. xxx_optimized_float_model.onnx to xxx_calibrated_model.onnx

      • This stage introduces Fake Quantization simulation, according to the quant_config you configured, simulate the operators to the corresponding bit widths, and this process may introduce numerical errors.

      • If a decrease in accuracy is observed for the xxx_calibrated_model.onnx:

    2. xxx_calibrated_model.onnx to xxx_ptq_model.onnx

      • In this stage, the fixed-point processing we perform on lookup table operators may introduce errors.

      • If accuracy degradation is observed in the actual test for the xxx_ptq_model.onnx, please contact Horizon Robotics technical support personnel for problem analysis.

    1. xxx_ptq_model.onnx to xxx_quantized_model.bc

      • Converting an ONNX model to an HBIR model (bc format) should be logically equivalent, but operator fixed-point processing may introduce further errors.

      • If accuracy degradation is observed for the xxx_quantized_model.bc, please contact Horizon Robotics technical support personnel for problem analysis.

    1. xxx_quantized_model.bc to xxx.hbm

      • The compilation process of converting an HBIR model (bc format) to an HBM model should theoretically not introduce numerical inconsistency.

      • If inconsistent outputs are observed between the HBIR model (bc format) and the HBM model, please contact Horizon Robotics technical support personnel for problem analysis.