#PyrDown

The Pyrdowm operator is used to obtain different resolution representations of the image. There are currently two types of interpolation supported by this operator: Gaussian interpolation and bilinear interpolation. The pyramids corresponding to the two interpolation methods are called Gaussian and bilinear pyramids, respectively.

• Gaussian pyramid downsamples the picture through processes such as Gaussian smoothing and sampling to produce an image whose length and width are half the size of the original.
• The bilinear pyramid directly generates a new image with a length and width half the size of the previous layer image of the current layer through bilinear interpolation, downsampling and other processes, and supports 10-bit and 12-bit image processing.
• The pyramid supports 5 levels of computation.

#Operator Effect

Input Image	Parameter	Output Image
	levels = 5 interpolation = HB_VP_INTER_GAUSSIAN

#Principle

Based on the principle of downsampling, there are two methods: Gaussian downsampling and bilinear downsampling.

The implementation process of downsampling with Gaussian pyramid is to first perform Gaussian smoothing on the original image and then remove the even rows and columns to finally get the target image.

The Gaussian filter kernel used in the Gaussian smoothing stage is as follows:

$\begin{aligned}kernel = \frac{1}{256}\begin{bmatrix}1 & 4 & 6 & 4 & 1 \\4 & 16 & 24 & 16 & 4 \\6 & 24 & 36 & 24 & 6 \\4 & 16 & 24 & 16 & 4 \\1 & 4 & 6 & 4 & 1 \end{bmatrix}\end{aligned}$

Bilinear pyramid uses bilinear interpolation to accomplish the downsampling calculation of the image, the principle of bilinear interpolation is shown in the figure below:

In the interpolation process, it is assumed that the interpolated pixel point to be output is P, and the distance between the four to-be-interpolated source points P1,P2,P3,P4 adjacent to P is one. Since the interpolation method is linear, the order of interpolation in the horizontal and vertical directions does not affect the final result. The specific interpolation point formula is as follows:

$P_{12} = (1 - a) * P_1 + a * P_2$

$P_{34} = (1 - a) * P_3 + a * P_4$

$P = (1 - a) * (1 - b) * P_1 + a * (1 - b) * P_2 + (1 - a) * b * P_3 + a * b * P_4$

The coefficients of the coordinates of each point in Eq. can be considered as the weight values of the interpolated source points for the output pixel points. The pyramid interface presented in this operator uses a bilinear interpolation defaulting to equal distances from the output point to each interpolated source point. The input image data is downsampled by one-half of its aspect size. After obtaining a 2*2 image region data, the corresponding homogenization is performed to obtain a resultant pixel.

The maximum number of layers for the op is up to 5 layers. The width and height of the image size of each layer is 1/2 of the previous layer. The specific calculations are as follows:

$BL_{o}(i,j) = \left( \text{SRC}(2*i, 2*j) + \text{SRC}(2*i + 1, 2*j) + \text{SRC}(2*i, 2*j + 1) + \text{SRC}(2*i + 1, 2*j + 1) + 2 \right) >> 2$

Among them, SRC is the input image pixel, i, j are the pixel coordinates of the output bl layer, and BL is the bilinear downsampling result of the current layer output.

#API Interface

int32_t hbVPPyrDown(hbUCPTaskHandle_t *taskHandle,
                    hbVPImage *dstImgs,
                    hbVPImage const *srcImg
                    hbVPPymParam const *pymCfg);

For detailed interface information, please refer to hbVPPyrDown.

#Usage

// Include the header
#include "hobot/hb_ucp.h"
#include "hobot/vp/hb_vp.h"
#include "hobot/vp/hb_vp_pyr_down.h"

// init Image, allocate memory for image data
hbUCPSysMem src_mem;
hbUCPMallocCached(&src_mem, src_stride * src_height, 0);
hbVPImage src_img{HB_VP_IMAGE_FORMAT_Y,
                  HB_VP_IMAGE_TYPE_U8C1,
                  src_width,
                  src_height,
                  src_stride,
                  src_mem.virAddr,
                  src_mem.phyAddr,
                  nullptr,
                  0,
                  0};

int layers = 5;
std::vector<hbUCPSysMem> dst_mems(layers);
std::vector<hbVPImage> dst_imgs(layers);
volatile int32_t dst_width = src_width;
volatile int32_t dst_height = src_height;
volatile int32_t dst_stride = src_stride;
for (int32_t i{0}; i < layers; i++) {
  dst_width = dst_width / 2;
  dst_height = dst_height / 2;
  dst_stride = dst_stride / 2;
  hbUCPMallocCached(&dst_mems[i], dst_stride * dst_height, 0);
  dst_imgs[i] = {HB_VP_IMAGE_FORMAT_Y,
                 HB_VP_IMAGE_TYPE_U8C1,
                 dst_width,
                 dst_height,
                 dst_stride,
                 dst_mems[i].virAddr,
                 dst_mems[i].phyAddr,
                 nullptr,
                 0,
                 0};
}

// init pyramid config
hbVPPymParam pym_cfg{};
pym_cfg.interpolation = HB_VP_INTER_GAUSSIAN;
pym_cfg.levels = layers;

// init task handle, schedule param and pyramid configuration
hbUCPTaskHandle_t task_handle{nullptr};
hbUCPSchedParam sched_param;
HB_UCP_INITIALIZE_SCHED_PARAM(&sched_param);
sched_param.backend = HB_UCP_DSP_CORE_0;

// create task
hbVPPyrDown(&task_handle, dst_imgs.data(), &src_img, &pym_cfg);

// submit task
hbUCPSubmitTask(task_handle, &sched_param);

// wait for task done 
hbUCPWaitTaskDone(task_handle, 0);

// release task handle
hbUCPReleaseTask(task_handle);

// release memory
hbUCPFree(&src_mem);
for (int32_t i{0}; i < layers; i++) {
  hbUCPFree(&dst_mems[i]);
}