Model Compilation and Performance Evaluation

(Optional) Model Modification

Before the model compilation, we also support modify the board-side deployable model, the common operations and the calling API interfaces are as follows:

After the model export and before convert:
1. Batch input splitting is performed by calling the insert_split() interface.
2. Insert the image preprocessing node to the model:
  
  a. The layout needs to be adjusted to NHWC for subsequent operations, which is performed by calling the insert_transpose() interface.
  
  b. Image normalization is performed by calling the insert_image_preprocess() interface.
  
  c. Color conversion (typically nv12 input for board-side deployment) is performed by calling the insert_image_convert() interface.
  
  d. Configuration of the input as a resizer input to support roi-based keying and scaling is performed by calling the insert_roi_resize() interface.
3. Adjustment the input and output data layout is performed by calling the insert_transpose() interface.
After the model convert and before compile, remove operators (Quantize/Dequantize/Cast, etc.) by calling the remove_io_op() interface.

For the specific steps of model modification, you can refer to the introduction in the Model Modification section. For detailed instructions on calling the above API interfaces, you can refer to the introduction in the HBDK Tool API Reference section.

Model Compilation and Performance Evaluation and Visualization

After testing the accuracy of the fixed-point model and confirming that it meets the requirements, the model can be compiled, performance tested and visualized.

Attention

The model used for perf should be calibrated at least once (with no limit on the number of steps) to ensure that the statistics in the model match the actual situation, otherwise it may cause inaccurate perf results.
The compile_opt supports configuration values of 0, 1, and 2. The higher the level, the faster the on-board execution speed of the compiled model, but the slower the compilation process. For performance evaluation, it is recommended that you set it to the default value of 2.
When the deviation between the actual computation time and the theoretical computation time of a certain OP is larger than the value your specified compile_advice, the relevant log will be printed, including information about the change in time, the shape and padding ratio before and after data alignment, etc. The unit is microseconds.

You can use The hb_analyzer Tool to perform performance analysis on HBIR models at different stages as well as HBM models. For detailed usage instructions and guidance on interpreting the analysis report, please refer to the corresponding section.

Alternatively, you can perform performance analysis based on hbm_perf using the following code snippet.

import os
from hbdk4.compiler import compile, hbm_perf

hbir_quantized_model = "quantized_model.bc"
model_path = "target_dir"

# Model compile
compile(
    hbir_quantized_model,
    os.path.join(model_path, "model.hbm"),
    "nash-e",
    opt=2,
    advice=0.0
)

# Model performance evaluation
hbm_perf(
    os.path.join(model_path, "model.hbm"),
    output_dir=model_path,
)

After successful execution, basic information such as the model's FPS will be printed in the terminal, and meanwhile, the static performance evaluation files for the model will be generated in the directory where you configured:

|-- model.html        # Static performance evaluation files (better readability)
|-- model.json        # Static performance evaluation file

You can select either the model.html or the model.json to view the static performance data for the BPU part.

If you configure the development board parameter remote_ip when calling hbm_perf, it will remotely connect to the development board for performance evaluation, and similarly generate the performance evaluation file for the model in the directory where the current API interface is called.

Interpretation of the metrics in the performance evaluation file can refer to the Metric Interpretation of Simulation Performance Evaluation section. If you find that the model performance evaluation results do not meet your expectations, you can refer to section Model Performance Optimization to perform performance tuning.

The above is the simulation-based performance evaluation on the development machine. If you want to conduct on-board actual performance measurement, please refer to the introduction in the Actual Measurement of Model Performance Data on the Board Side section, the steps are the same.

If you also need the visualized model, please refer to the following:

from hbdk4.compiler import visualize

# Model Visualization.
visualize(hbir_quantized_model, "mobilenetv2_cifar10.onnx")

For detailed instructions on calling the above API interfaces, you can refer to the introduction in the HBDK Tool API Reference section.

#Model Compilation and Performance Evaluation

#(Optional) Model Modification

#Model Compilation and Performance Evaluation and Visualization

Model Compilation and Performance Evaluation

(Optional) Model Modification

Model Compilation and Performance Evaluation and Visualization