• English
  • 3.8.1

    • Model Compilation and Performance Analysis Tool

    HAT is capable of compiling fixed-point models to make the compiled Pytorch models suitable for on-board running. You can choose to compile the model as described in the next section.

    Usage

    This method uses the script tools/compile_perf_hbir.py with the following command.

    python3 tools/compile_perf_hbir.py -c {$CONFIG_FILE}

    For a detailed introduction to the configuration file, please refer to the [Configuration] section [config configuration] (../config/keywords).

    Attention

    If you want to deploy the model provided by the reference algorithm package on nash-p, nash-b, or nash-h, you need to pass the corresponding march parameter.

    Parameters Introduction

    ParameterDescription
    --config, -cDirectory of the config file.
    --optWhether only export pre compile hbir. The selectable values are True and False.

    Usage Example

    This command will get the compilation-related configurations from the hbir_compiler field in the config file, as illustrated by the following example:

    import horizon_plugin_pytorch

hbir_compiler = dict(
    march=March.NASH_M,  # Options: NASH_E, NASH_M, NASH_P
    opt=2,  # Optimization level: 0, 1, 2 (from low to high)
    jobs=32,  # Multi-threaded compilation
    debug=False,  # Whether to enable debug mode
    only_export=args.only_export,  # Whether to only export the pre-compiled HBIR
    layer_details=True,  # Whether to output performance details for each layer
    input_source=["pyramid"],  # Input source when deploying the model
    model_path="./qat.bc",  # Path to the QAT model
    out_path="tmp_compile/model.hbm",  # Path to the compiled model file
    hbdk3_compatible_mode=False,  # Whether to compile in compatibility mode for HBM
    transpose_dim,  # Reorder the dimensions of the input/output tensor
    split_dim,  # Split the input tensor in pyramid format
    transpose_dim=dict(
        inputs={
            "1": [0, 2, 3, 1],
        }
    ),      # Reorder tensor dims of inputs/outputs.
    split_dim=dict(
        inputs={
            "0": [0, 6],
        }
    ),      # Split input tensors
    enable_vpu=False, # Whether to enable VPU
)

    After the compilation is completed, the generated model.hbm file will be saved to the pre-set storage path for the compiled model files.