In the Softmax operator sample provided by Horizon, it is demonstrated how to implement the functional encapsulation of DSP custom operators by scheduling the UCP framework. You can obtain the sample source code at samples/ucp_tutorial/custom_operator/dsp_sample in the OE package for synchronous reading and understanding.
This section mainly introduces the process of DSP operator development by taking DSP Softmax operator development as an example, which corresponds to the samples/ucp_tutorial/custom_operator/dsp_sample/dsp_code/softmax/softmax_ivp.cc part in Horizon OE.
The complete operator development is divided into the following three steps:
Calling UCP API, which is mainly responsible for task initiation and computing resource allocation.
In this section, we will show you how to use the UCP interface to apply for memory resources and initiate tasks. Taking the Softmax example, since the UCP interface only supports the two parameters of input and output, if you need to pass other parameters to the DSP, you need to perform a layer of encapsulation, pass the encapsulated "input" to the interface, and then the DSP obtains the parameters from the encapsulated "input".
In the following sample, the encapsulated "input" includes three fields, data_size is the data size, 100000 in the sample, the input field is the data address sent to the dsp, and the run_tcm_opt field indicates whether to use tcm optimization implementation.
// softmax op paramtypedef struct { int32_t data_size; uint64_t input; uint8_t run_tcm_opt;} hbDSPSoftmaxParam;// #define DEBUG_RESULTint32_t test_softmax_op(int32_t argc, char **argv) { LOGI("softmax op begin"); int32_t size = 100000; int32_t data_size = size * sizeof(float32_t); hbUCPSysMem input_mem, output_mem, output_opt_mem; // prepare input data hbUCPSysMem input_data; HB_CHECK_SUCCESS(hbUCPMalloc(&input_data, data_size, 0), "hbUCPMalloc input_data failed"); HB_CHECK_SUCCESS(hbDSPAddrMap(&input_data, &input_data), "Failed to hbDSPAddrMap input_data"); float32_t *data_ptr = static_cast<float32_t *>(input_data.virAddr); for (uint32_t i = 0U; i < size; ++i) { data_ptr[i] = 1 + i / float32_t(size); } HB_CHECK_SUCCESS(hbUCPMalloc(&output_mem, data_size, 0), "hbUCPMalloc output_mem failed"); HB_CHECK_SUCCESS(hbDSPAddrMap(&output_mem, &output_mem), "Failed to hbDSPAddrMap output_mem"); HB_CHECK_SUCCESS(hbUCPMalloc(&output_opt_mem, data_size, 0), "hbUCPMalloc output_opt_mem failed"); HB_CHECK_SUCCESS(hbDSPAddrMap(&output_opt_mem, &output_opt_mem), "Failed to hbDSPAddrMap output_opt_mem"); HB_CHECK_SUCCESS(hbUCPMalloc(&input_mem, sizeof(hbDSPSoftmaxParam), 0), "hbUCPMalloc input_mem failed"); HB_CHECK_SUCCESS(hbDSPAddrMap(&input_mem, &input_mem), "Failed to hbDSPAddrMap input_mem"); hbDSPSoftmaxParam *ptr = static_cast<hbDSPSoftmaxParam *>(input_mem.virAddr); ptr->data_size = data_size; ptr->input = input_data.phyAddr; ptr->run_tcm_opt = 0; // ctrl param hbUCPSchedParam sched_param; HB_UCP_INITIALIZE_SCHED_PARAM(&sched_param); sched_param.backend = HB_UCP_DSP_CORE_0; sched_param.priority = 0; // run without ping-pong hbUCPTaskHandle_t task{nullptr}; HB_CHECK_SUCCESS(hbDSPRpcV2(&task, &input_mem, &output_mem, HB_DSP_RPC_CMD_NN_B), "hbDSPRpcV2 failed"); HB_CHECK_SUCCESS(hbUCPSubmitTask(task, &sched_param), "hbUCPSubmitTask failed"); HB_CHECK_SUCCESS(hbUCPWaitTaskDone(task, 0), "hbUCPWaitTaskDone failed"); HB_CHECK_SUCCESS(hbUCPReleaseTask(task), "hbUCPReleaseTask failed"); // run without ping-pong ptr->run_tcm_opt = 1; task = nullptr; HB_CHECK_SUCCESS(hbDSPRpcV2(&task, &input_mem, &output_opt_mem, HB_DSP_RPC_CMD_NN_B), "hbDSPRpcV2 failed"); HB_CHECK_SUCCESS(hbUCPSubmitTask(task, &sched_param), "hbUCPSubmitTask failed"); HB_CHECK_SUCCESS(hbUCPWaitTaskDone(task, 0), "hbUCPWaitTaskDone failed"); HB_CHECK_SUCCESS(hbUCPReleaseTask(task), "hbUCPReleaseTask failed"); float32_t *dst_data_ptr = static_cast<float32_t *>(output_mem.virAddr); float32_t *dst_opt_data_ptr = static_cast<float32_t *>(output_opt_mem.virAddr); bool flag{true}; for (uint32_t i = 0U; i < size; ++i) { if (std::abs(dst_data_ptr[i] - dst_opt_data_ptr[i]) / std::abs(dst_data_ptr[i]) > 1e-6) { LOGE("opt vs non-opt result is diff {}, dst {} opt {}", i, dst_data_ptr[i], dst_opt_data_ptr[i]); flag = false; break; } } if (flag) { LOGI("opt vs non-opt result is same"); } hbDSPAddrUnmap(&input_mem); hbUCPFree(&input_mem); hbDSPAddrUnmap(&output_mem); hbUCPFree(&output_mem); hbDSPAddrUnmap(&output_opt_mem); hbUCPFree(&output_opt_mem); hbDSPAddrUnmap(&input_data); hbUCPFree(&input_data); LOGI("softmax op finish"); return 0;}
This section will introduce how to implement the four basic operations mentioned in the previous section to realize the DSP Softmax operator.
Cadence implements some basic mathematical operations to facilitate your development. You can get the source code from the basic examples of Cadence, or directly get the compiled dependency library dsp_math from Horizon.
In order to fully utilize the hardware performance, you need to understand the DSP features and use these features (VLIW, SIMD). When developing, you can refer to the basic operations that Cadence has already implemented.
The division operation can be transformed into a multiplication operation, and the multiplication operation is easier to implement and has better performance.
The above Softmax implementation directly accesses DDR. Since J6 is not configured with vector cache, direct access to DDR memory is very time-consuming. A common optimization implementation is to divide the data into tiles, move them to TCM (TCM has the same latency as cache) through idma, and directly access TCM for calculation. In addition, in order to parallelize the calculation with the idma transfer operation, the DSP is equipped with two TCMs to implement ping pong DMA operations, as shown in the figure below.
The reference code for the implementation of ping pong DMA is as follows:
After the operator development is completed, register the hb_dsp_softmax operator by calling the hb_dsp_register_fn interface. After registration is completed, compile the DSP image.
typedef int (*handle_fn)(void *input, void *output, void *tm);/** * register custom op, should be called before hb_dsp_start * @param[in] latency: op latency(microseconds), if no latency info, set 0 * @return 0 if success, return defined error code otherwise */int hb_dsp_register_fn(int cmd, handle_fn handle, int latency);
Note
If it is x86 simulation running, there is no need to start the image. If it is running on the board, you need to execute the script dsp_deploy.sh to start the image.
