#Performance Evaluation and Tuning

After completing the quantization and compilation of the model to obtain the hbm model deployed on the board side, we not only support the static performance of the BPU part of the model to be predicted on the developer side, but also provide executable tools on the board side that do not require any code development to help you quickly evaluate the dynamic performance of the model.

#Static Performance Evaluation

• If you are quantizing the model through the PTQ path and using the command line tool, the hb_compile tool will generate a static performance evaluation file (*.html & *.json) by default when it compiles the model, and at the same time, you can also evaluate the static performance of the model by calling the hbm_perf interface. The reference commands are as follows:

  from hbdk4.compiler import hbm_perf
  hbm_perf("model.hbm")

  After successful execution, basic information such as model FPS will be printed within the terminal, and at the same time, a static performance evaluation file (model.html & model.json) will be generated for the model in the directory where the API interface is currently called. If you need to specify the path to the static performance evaluation file, you can refer to the following command:

  from hbdk4.compiler import hbm_perf
  hbm_perf("model.hbm", output_dir="target_dir")

• If you are quantizing the model through the QAT path, you can just call the hbm_perf interface directly to perform a static performance evaluation of the model in the same way as described above.

#Dynamic Performance Evaluation

Once the static performance of the model is as expected, we also support the further use of the hrt_model_exec tool to empirically measure the dynamic performance of the model due to the presence of dynamic properties in the Pyramid and Resizer input models. The reference command is as follows:

hrt_model_exec perf --model_file model.hbm

For dynamic inputs, the requirement of stride is 32-aligned in the W direction, and the detailed calculation can be referred to the relevant contents of the introduction of UCP-Dynamic Inputs.

Assume that the input size of tensor valid_shape is (1,112,112,2), tensor_type=HB_DNN_TENSOR_TYPE_U8, stride= (-1,-1,2,1), then stride is calculated as follows:

// stride[3]=sizeof(tensor_type)                 -> 1
// stride[2]=stride[3]*valid_shape[3]            -> 2
// stride[1]=ALIGN_32(stride[2]*valid_shape[2])  -> 2*112=224
// stride[0]=stride[1]*valid_shape[1]            -> 224*112=25088

#define ALIGN_32(value) ((value + (32-1)) & ~(32-1))

For example, if the Pyramid input model with Y=(1,224,224,1) and UV=(1,112,112,2), the reference command would be as follows:

hrt_model_exec perf --model_file pyramid.hbm --input_stride="50176,224,1,1;25088,224,2,1"

For the Resizer input model, the tool will specify the default ROI internally, and the default value is [0,0,127,127], so the default value can not meet the requirements of your model, you can refer to the following command to specify:

hrt_model_exec perf --model_file resizer.hbm --input_file=test.jpg,test.jpg,roi.txt  --input_img_properties=Y,UV

The roi.txt is specified within the ROI (Region of Interest) by the order of the four region side coordinates left, top, right, bottom, separated by spaces, for example:

0 0 200 200

#Performance Tuning Recommendations

Based on the performance evaluation and analysis above, you may find that the performance result is not as expected, at this time you can refer to the Model Performance Tuning section, we provide suggestions and measures to improve the performance of the model for tuning attempts.