#ISP

The ISP operator is used to convert a RAW image rich in original image data into a YUV image which is more widely used in video processing.

#Operator Effect

Input Image	Parameter	Output Image
	bufNum = 3 bufCached = 0 backend = HB_UCP_ISP_CORE_0 width = 1920 height = 1080

#Principle

The ISP processes RAW images, including but not limited to the following submodules:

• BLC (Black Level Correction)：The sensor circuit itself has dark current, which causes the pixel unit to have a certain output voltage when there is no light. BLC adjusts the black level offset in the image sensor output so that the dark area of ​​the image reflects the brightness of the scene more realistically.

• LSC (Lens Shading Correction)：In the actual image capture process, the uneven refraction of light by the lens will cause shadows to appear around the lens. LSC corrects the image brightness and color. In the actual image processing process, the light attenuation characteristics of the lens are measured, a correction model is generated and applied to each pixel of the image, and the image brightness and color distribution are made more uniform by increasing the brightness of edge pixels and adjusting the intensity of color channels.

• DPC (Defect Pixel Correction)：Due to defects in the sensor photosensitive array process or errors in the conversion of light signals, the pixel values ​​in the image are inaccurate. DPC usually uses neighboring interpolation and pattern matching to make the bad pixels consistent with the surrounding normal pixels.

• 2DNR/3DNR (2D/3D Noise Reduce)：Image noise is usually generated by the imaging device itself and the external environment during the acquisition, transmission and recording of signals. ISP includes 2D and 3D noise reduction modules, which perform spatial noise reduction on single-frame images and use time domain filtering technology to reduce temporal noise.

• Demosaic：Converting raw pixel Data captured by a monochrome image sensor (such as a Bayer filter array) into a full-color image. Most digital cameras and image sensors use a Bayer filter, which separates the sensor's pixels into three color channels: red, green, and blue. The process of demosaicing is to recover the full RGB color image from these monochrome sampled data.

• CSC (Color space conversion)：The CSC module converts the image data format from RGB to YUV. Color space conversion is usually achieved through matrix transformation or nonlinear transformation.

#API Interface

int32_t hbVPCreateISPContext(hbVPISPContext *context,
                             hbVPISPCtxParam const *ispCtxParam);

int32_t hbVPISP(hbUCPTaskHandle_t *taskHandle, 
                hbVPImage const *srcImg,
                hbVPISPContext context);

int32_t hbVPGetISPOutputBuffer(hbVPImage *outImg, 
                               hbUCPTaskHandle_t handle);

int32_t hbVPReleaseISPContext(hbVPISPContext context);

For detailed interface information, please refer to hbVPISP。

#Usage

// include the header
#include "hobot/hb_ucp.h"
#include "hobot/vp/hb_vp.h"
#include "hobot/vp/hb_vp_isp.h"

// init Image, allocate memory for image data
hbUCPSysMem src_mem;
int32_t src_width = 1920;
int32_t src_height = 1080;
int32_t raw_size{src_width * src_height * 3 / 2};
hbUCPMalloc(&src_mem, raw_size, 0);

hbVPImage src_img{HB_VP_IMAGE_FORMAT_Y,
                  HB_VP_IMAGE_TYPE_U12C1,
                  src_width,
                  src_height,
                  0,
                  src_mem.virAddr,
                  src_mem.phyAddr,
                  0,
                  0,
                  0};

int32_t dst_width{src_width};
int32_t dst_height{src_height};

hbVPImage dst_img;

// init isp config
hbVPISPCtxParam isp_ctx_param = {3, 0, HB_UCP_ISP_CORE_0,
                                  dst_width, dst_height};

// init task handle, schedule param and isp context
hbUCPTaskHandle_t task_handle{nullptr};
hbVPISPContext isp_ctx{nullptr};
hbUCPSchedParam sched_param;
sched_param.backend = HB_UCP_ISP_CORE_0;
sched_param.priority = 0;

// create isp context
hbVPCreateISPContext(&isp_ctx, &isp_ctx_param);

// create task
hbVPISP(&task_handle, &src_img, isp_ctx);

// submit task
hbUCPSubmitTask(task_handle, &sched_param);

// wait for task done 
hbUCPWaitTaskDone(task_handle, 0);

// get output buffer
hbVPGetISPOutputBuffer(&dst_img, task_handle);

// release task handle
hbUCPReleaseTask(task_handle);

// release isp context
hbVPReleaseISPContext(isp_ctx);

// release memory
hbUCPFree(&src_mem);