• English
  • 3.5.0

    • Model Inference

    hbDNNInferV2

    int32_t hbDNNInferV2(hbUCPTaskHandle_t *taskHandle,
                     hbDNNTensor *output,
                     hbDNNTensor const *input,
                     hbDNNHandle_t dnnHandle);

    Create synchronous/asynchronous inference tasks based on input parameters. For the asynchronous inference task, the caller can use the returned taskHandle across functions and threads.

    • Parameter
      • [out] taskHandle Task handle pointer.
      • [in/out] output Output of the inference task.
      • [in] input Input of the inference task.
      • [in] dnnHandle DNN handle pointer.
    • Return Value
      • Return 0 means the API was successfully executed, otherwise the execution failed.
    Note
    1. If taskHandle is set to nullptr, a synchronous task will be created, as returned by the interface, and the interface will be complete. Synchronous tasks do not support multi-core model inference.
    2. If *taskHandle is set to nullptr, an asynchronous task will be created, and the taskHandle returned by the interface can be used for subsequent blocking or callbacking.
    3. If *taskHandle is not null and points to a previously created but uncommitted task, a new task will be created and added to it. Multi-core tasks do not support task addition operations.

    Up to 32 coexisting model tasks are supported.

    hbDNNGetTaskOutputTensorProperties

    int32_t hbDNNGetTaskOutputTensorProperties(hbDNNTensorProperties *properties,
                                           hbUCPTaskHandle_t taskHandle,
                                           int32_t subTaskIndex,
                                           int32_t outputIndex);

    Get dynamic output property information of the task that taskHandle points to.

    • Parameter
      • [out] properties Info of the input tensor.
      • [in] taskHandle Task handle pointer.
      • [in] subTaskIndex Subtask index.
      • [in] outputIndex Index of the output tensor of the subtask.
    • Return Value
      • Return 0 means the API was successfully executed, otherwise the execution failed.
    Note

    1. If the model has dynamic output, you need to use this interface to obtain the dynamic attribute information of the task output for subsequent processing. The non-dynamic output tensor information is the same as that obtained by hbDNNGetOutputTensorProperties.

    2. If the task pointed to by taskHandle has multiple subtasks, subTaskIndex is used to specify the subtask number.

    3. If the model contains dynamic output, please do not use the synchronous task execution method during inference, because the synchronously executed task cannot obtain the taskHandle.