#The hrt_model_exec Tool Introduction

The hrt_model_exec is a model execution tool that can evaluate the inference performance of the model and get the model information directly on the development board.

On one hand, it allows you to get a realistic understanding of the model's real performance; On the other hand, it also helps you to learn the speed limit that the model can achieve, which is useful information in application tuning.

The hrt_model_exec tool source code is located in the samples/ucp_tutorial/tools/hrt_model_exec path of the horizon_j6_open_explorer publication. The structure is as follows:

├── include               # Header file
├── src                   # Source code
├── build.sh              # Compile script
├── build_x86.sh          # Compile to produce x86 tools
├── build_aarch64.sh      # Compile to produce aarch64 tools for the Linux system
├── build_qnx.sh          # Compile to produce aarch64 tools for the QNX system
├── CMakeLists.txt        
├── README.md
├── script                       
│   └── run_hrt_model_exec.sh    # The original aarch64 tool running script, which will be downloaded to the output_shared_J6_aarch64/script directory after compilation.
└── script_x86                   
    └── run_hrt_model_exec.sh    # The original x86 tool running script, which will be downloaded to the output_shared_J6_x86/script_x86 directory after compilation.

#Running Process

#Building Method

There is a pre-configured compilation script build.sh in the ucp_tutorial/tools/hrt_model_exec directory. The options -a x86, -a aarch64-qnx and -a aarch64 support three compilation modes respectively. You can use this script and specify the compilation options for compilation. In addition, the directory also contains three compilation scripts, build_aarch64.sh, build_qnx.sh and build_x86.sh, which correspond to three compilation options respectively. Compiling with these scripts is equivalent to using the build.sh script and specifying the compilation options.

# Build board-side hrt_model_exec tools for linux system
bash -ex build_aarch64.sh
# Build x86-side hrt_model_exec tools
bash -ex build_x86.sh
# Build board-side hrt_model_exec tools for qnx system
bash -ex build_qnx.sh

#Execute the Method

Take Linux environment as an example, after building board-side hrt_model_exec tools, the output_shared_J6_aarch64 folder will be generated. You can use this tool by copying the folder to the board environment and executing output_shared_J6_aarch64/script/run_hrt_model_exec.sh.

After building x86-side hrt_model_exec tools, the output_shared_J6_x86 folder will be generated. You can use this tool on the x86 environment and executing output_shared_J6_x86/script_x86/run_hrt_model_exec.sh.

The run_hrt_model_exec.sh script is divided into two parts: setting environment variables and getting model information and inferring the model.

# Set environment variables
# arch represents the architecture type, aarch64 or x86
arch=aarch64
bin=../$arch/bin/hrt_model_exec
lib=../$arch/lib/
export LD_LIBRARY_PATH=${lib}:${LD_LIBRARY_PATH}

# Get model information
${bin} model_info --model_file=xxx.hbm

# Infer the model 
${bin} infer --model_file=xxx.hbm --input_file=xxx.bin

# Get model performance
${bin} perf --model_file=xxx.hbm --frame_count=200

#Usage Scenarios

The hrt_model_exec tool supports three usage scenarios: view the model information, model inference, model performance evaluation.

• View the Model Information: Get model information, such as model input and output information, etc.

• Model Inference: Perform model inference and get model inference results.

• Model Performance Evaluation: Perform model performance analysis and obtain performance analysis results.