Model Compilation

HAT is capable of compiling fixed-point models to make the compiled Pytorch models suitable for on-board running. You can choose to compile the model as described in the next section.

Usage

This method uses the script tools/compile_perf_hbir.py with the following command.

python3 tools/compile_perf_hbir.py -c {$CONFIG_FILE}

This command will get the compilation-related configurations from the hbir_compiler field in the config file, as illustrated by the following example:

import horizon_plugin_pytorch

hbir_compiler = dict(
    march=March.NASH_M,  # Options: NASH_E, NASH_M, NASH_P
    opt=2,  # Optimization level: 0, 1, 2 (from low to high)
    jobs=32,  # Multi-threaded compilation
    debug=False,  # Whether to enable debug mode
    only_export=args.only_export,  # Whether to only export the pre-compiled HBIR
    layer_details=True,  # Whether to output performance details for each layer
    input_source=["pyramid"],  # Input source when deploying the model
    model_path="./qat.bc",  # Path to the QAT model
    out_path="tmp_compile/model.hbm",  # Path to the compiled model file
    hbdk3_compatible_mode=False,  # Whether to compile in compatibility mode for HBM
)

After the compilation is completed, the generated model.hbm file will be saved to the pre-set storage path for the compiled model files.

#Model Compilation

#Usage

Model Compilation

Usage