• English
  • 3.10.0

    • X86 Simulation

    The toolchain supports application development capabilities in an X86 simulation environment, allowing you to quickly complete program debugging and verify computational results in the simulation environment when using a development board is not convenient.

    C++ Inference

    Develop validation code using the UCP inference interface, which supports inference of quantized.bc model and hbm model. The UCP simulation library interface is fully consistent with the embedded interface, except for using different compilation configurations. For an example of how to use the UCP inference interface, please refer to Model Inference Application Development Guide.

    Command-Line Tool

    Use the X86 version of the hrt_model_exec tool to view model information and perform single-frame inference, supporting inference of quantized.bc model and hbm model. For a detailed introduction and usage examples of hrt_model_exec, please refer to The hrt_model_exec Tool Introduction.

    Python Inference

    • Use the HBRuntime interface encapsulated by the horizon_tc_ui package, which supports inference of quantized.bc model and hbm model. For inference examples, please refer to The HBRuntime Inference Library.

    • Use the hbdk API inference interface, which supports inference of quantized.bc model and hbm model. For a detailed introduction and usage examples of the API, please refer to HBDK Tool API Reference - model inference.

    Note

    • Prior to performing inference on an hbm model in an x86 simulation environment using the UCP inference interface or the hrt_model_exec tool, the HB_UCP_SIM_PLATFORM_TYPE environment variable must be set to specify the target BPU architecture to be simulated. For detailed instructions on configuring this environment variable, please refer to Environment Variable.

    • When performing inference on quantized.bc using the UCP simulation library, GPU acceleration is supported. For configuration details, please refer to HBIR Model Inference Instructions.

    quantized.bc Simulation Acceleration and Result Consistency

    The toolchain supports instruction simulation of hbm models in an X86 environment. However, because this approach provides relatively low simulation performance, using quantized.bc models is recommended. For the parts of a quantized.bc model executed on the BPU, the X86 simulation output is bit-exact with the on-device inference output of the corresponding hbm model.

    When simulating a quantized.bc model, note the following acceleration capabilities and performance configurations:

    • The b30.Conv2d operator supports CPU hardware acceleration based on avx512_core_vnni and avx512_core. At runtime, the acceleration method is selected automatically according to the instruction sets supported by the host CPU. Set the DNNL_VERBOSE=1 environment variable to view detailed runtime information.

    • Use the HBTL_THREAD_NUM environment variable to configure the number of OMP threads used for simulation inference. The default value is 16.

    • On the J6B, J6H, J6P, and S6P computing platforms, when the b30.Conv2d operator produces float16 or float32 output, the simulation speed decreases to some extent to ensure that the X86 simulation output remains bit-exact with the on-device inference output of the corresponding hbm model.