This tutorial focuses on how to use HAT to train a PointPillars model on the radar point cloud dataset KITTI-3DObject from scratch, including floating-point, quantized, and fixed-point models.
If you just want to simply train the PointPillars model, then you can read this section first.
Similar to other tasks, HAT performs all training tasks and evaluation tasks in the form of tools + config.
After preparing the original dataset, take the following process to complete the whole training process.
In order to create KITTI point cloud data, you need to load the original point cloud data and generate the associated data annotation file containing the target labels and annotation boxes.
It is also necessary to generate the point cloud data for each individual training target for the KITTI dataset and store it in a .bin file in data/kitti/gt_database.
In addition, you need to generate a .pkl file containing data information for the training data or validation data.
Then, create the KITTI data by running the following commands:
Once the dataset is ready, you can start training the floating-point PointPillars detection network.
Before the network training starts, you can test the number of operations and the number of parameters of the network by using the following command:
The HAT algorithm package uses an ingenious registration mechanism that allows each training task to be started in the form of train.py plus a config file.
The train.py is a uniform training script and independent of the task.
The task we need to train, the dataset we need to use, and the hyperparameters we need to set for the training are all in the specified config file.
The config file provides key dicts such as model building and data reading.
After the model is trained, we can also verify the performance of the trained model.
Since we provide two stages of training process, Float, Calibration and QAT, we can verify the performance of the model trained in these stages.
Additionally, the following command can be used to verify the accuracy of a fixed point model, but it should be noted that hbir must be exported first:
The displayed accuracy is the real accuracy of the final int8 model.
Of course, this accuracy should be very close to the accuracy of the QAT verification stage.
In addition to the model validation described above, we offer an accuracy validation method that is identical to the board side, you can refer to the following:
The quantization training toolchain integrated in HAT is mainly prepared for the Horizon computing platform, so it is necessary to check and compile the quantized models.
We provide a model checking interface in HAT, which can be used to check whether the defined quantization model can run normally on the BPU before training:
In which, the type under model means the name of the defined model, and the remaining variables mean the other components of the model.
The advantage of defining the model in this way is that the structure can be easily replaced.
After starting, the training script calls the build_model interface to convert such a model of the dict type into a model of the torch.nn.Module type.
def build_model(cfg, default_args=None): if cfg is None: return None assert "type" in cfg, "type is need in model" return build_from_cfg(cfg, MODELS)
Like the definition of model, the data augmentation process is implemented by defining two dicts (data_loader and val_data_loader) in the config file, corresponding to the processing workflow of the training set and validation set, respectively.
Here we tak data_loader as an example:
Here, type directly uses the interface torch.utils.data.DataLoader that comes with pytorch, which means to combine the images with the size of batch_size together.
Here you may only need to pay attention to the dataset variable. The path data_path is the path we mentioned in the first part of the dataset preparation.
The transforms contains a series of data augmentation, while val_data_loader contains only point cloud Pillarization (Voxelization) and Reformat.
You can also implement your desired data augmentation operations by inserting a new dict in transforms.
The float_trainer defines our training approach in general, including the use of distributed_data_parallel_trainer, the number of epochs for model training, and the choice of optimizer.
Also, callbacks reflects the small strategies used by the model in the training process and the operations that you want to implement, including the transformation method of the learning rate (CyclicLrUpdater),
and the indicator (Validation), and the save operation (Checkpoint) of the model. Of course, if you have operations that you want the model to implement during the training,
you can also add it in this way (dict).
The float_trainer is responsible for concatenating the entire training logic, which is also responsible for model pretraining.
Note
If you need to reproduce the accuracy, it is best not to modify the training strategy in the config file. Otherwise, unexpected training situations may arise.
Through the above introductions, you should have a clear understanding of the role of the config file.
Then, through the training script mentioned above, a high accuracy pure floating-point detection model can be trained.
Of course, training a good detection model is not our ultimate goal, it is just a pre-training for our future training of fixed-point models.
When we have a floating-point model, we can start training the corresponding fixed-point model.
In the same way as the floating-point training, we can train a fixed-point model just by running the following script:
As you can see, our configuration file has not changed, except the stage type.
At this point, the training strategy we use comes from the qat_trainer and calibration_trainer in the config file.
When we train the quantized model, we need to set quantize=True.
At this moment, the corresponding floating-point model will be converted into a quantized model. The code is as follows:
For the key steps in quantization training, such as preparing floating-point models, operator replacement, inserting quantization and dequantization nodes, setting quantization parameters, and operator fusion,
please read the Quantized Awareness Training (QAT) section.
As we said before, quantization training is actually the finetuning on the basis of pure floating-point training.
Therefore, in quantization training, the initial learning rate is set to one-tenth of the floating-point training, the number of epochs for training is also greatly reduced,
and most importantly, when model is defined, the pretrained parameter needs to be set to the address of the pure floating-point model that has been trained.
After making these simple adjustments, we can start training our quantized model.
