The DSP sample package is developed based on the SLAM sample package provided by Cadence and can be run on a board or in simulation. This section will briefly introduce the SLAM DSP sample process.

#Principle Introduction

SLAM(simultaneous localization and mapping), also known as CML(Concurrent Mapping and Localization), is a concept used to locate in an environment and describe the current environment for route planning; it records information obtained by some form of perception and compares it with the current perception results to support the evaluation of actual positioning. The problem can be described as: if a robot is placed in an unknown position in an unknown environment, is there a way for the robot to gradually draw a complete map of the environment while deciding in which direction the robot should go.

#SLAM Implementation Process

The visual SLAM application sample is based on the open source ORB_SLAM2 implementation. However, the application only extracts the most basic functions and simplifies and modifies them in various functions. These modifications are designed to adapt to the SIMD implementation on the Vision DSP to improve loop performance and reduce frequent random accesses to system memory. In addition, additional functions for removing outliers have been added in various modules.

The SLAM application also supports "binocular stereo" and "RGBD" inputs. However, its approach is significantly different from that of ORB_SLAM2. ORB_SLAM2 uses additional information (right image in binocular stereo vision and depth information in RGBD) to reduce reprojection error or cost function. Instead, the application uses a proprietary method that leverages this additional information to perform outlier rejection.

The figure below shows the module diagram of the SLAM application. The SLAM application can accept monocular, binocular stereo or RGBD input. In case of RGBD input, grayscale monocular image and depth image are provided as input. Keypoints or corner points are then detected from the input image and descriptors are created for these keypoints, which are used for matching. In the case of binocular stereo input, depth information is generated from binocular stereo images. For monocular input, in the initial state, the SLAM application tries to find a set of two keyframes to obtain the initial 3D point set and pose. For RGBD and stereo input, 3D points are generated on the first frame and the application is initialized. Once the application is initialized, it enters the tracking state, estimating the camera pose through pose optimization in each frame using known 3D points and their observed projections. Depending on the conditions, keyframes (KFs) are inserted to generate new 3D point sets, and local bundle adjustments further optimize the estimated 3D points and poses.