This tutorial shows how to train a fixed-point 3D detection model using the HAT algorithm toolkit by using FCOS3D-efficientnetb0 as an example.
Before starting the quantized awareness training, namely, the fixed-point model training, you need to first train a pure floating-point model with high accuracy, by finetuning this pure floating-point model, you can train the fixed-point model quickly.
Let's start from training a pure floating-point FCOS3D-efficientnetb0 model.
If you just want to simply train the FCOS3D-efficientnetb0 model, then you can read this section first.
Similar to other tasks, HAT performs all training tasks and evaluation tasks in the form of tools + config.
After preparing the original dataset, take the following process to complete the whole training process.
Before starting to train the model, the first step is to prepare the dataset.
Here we use nuscenes to train FCOS3D. After unpacking The data directory structure is shown as below:
Also, to improve the training speed, we packaged the original jpg format dataset and converted it to the lmdb format.
The conversion can be successfully achieved by simply running the following script:
These two commands are for training dataset conversion and validation dataset conversion respectively. When the packing is done, the file structure of data should be as below:
tmp_data|-- nuscenes |-- train_lmdb |-- val_lmdb |-- meta
The above train_lmdb and val_lmdb are the packed training and validation datasets, which are also the final datasets read by the network.
Once datasets are ready, you can start training the floating-point FCOS3D-efficientnetb0 detection network.
If you simply want to start such a training task, just run the following command:
Since the HAT algorithm toolkit uses an ingenious registration mechanism, each training task can be started in the form of train.py plus a config file.
train.py is a uniform training script and independent of the task.
The task we need to train, the dataset we need to use, and the hyperparameters we need to set for the training are all in the specified config file.
The config file provides the key dict for model building, data reading, etc.
After the model is trained, we can also validate the performance of the trained model.
Since we provide three stages of training, float, calibration and qat, we can validate the performance of the trained model in these stages.
This displayed accuracy is the real accuracy of the final int8 model, which of course should be very close to the accuracy of the qat verification phase.
If you want to see the results of the trained model detecting a single image, we also provide scripts for single image prediction and visualization under our tools folder.
Run the following script:
The quantization training toolchain integrated in HAT is mainly prepared for the Horizon computing platform, so it is necessary to check and compile the quantized models.
We provide a model checking interface in HAT, which can be used to check whether the defined quantization model can run normally on the BPU before training:
After the training, you can use the compile_perf_hbir tool to compile the quantized model into a board-ready hbm file.
The compile tool can also predict the on-computing platform running performance.
Run the following script:
The type under model is the name of the defined model, and the remaining variables stand for the other components of the model.
By defining the model in this way, we can easily replace the structure we want.
Like the definition of model, the data augmentation process is implemented by defining two dicts, data_loader and val_data_loader, in the config file, corresponding to the processing process of the training and validation sets, respectively.
Take data_loader as an example:
A good training strategy is essential for training a model with high accuracy.
For each training task, the corresponding training strategy is also defined in the config file, as can be seen from the float_trainer variable.
The float_trainer defines the training approach in general, including the use of multi-card distributed training (distributed_data_parallel_trainer), the number of epochs for model training, and the choice of optimizer.
The callbacks reflects the small strategies used by the model during the training and the operations that you want to implement, including the way to transform the learning rate (WarmupStepLrUpdater), the metrics to validate the model during training (Validation), and the operations to save (Checkpoint) the model.
Of course, if you have your own operations that you want the model to implement during the training, you can also add them in this way using dict.
The above information may have give you a clearer understanding of the functions of the config file.
The training script mentioned earlier can help you train a pure floating point detection model wih high accuracy.
However, a well-trained detection model is not our goal, it is just a pretrain for us to train a fixed-point model later.
With a pure floating-point model in place, we can start training the corresponding fixed-point model.
Similar to the floating-point training, we can train the fixed-point model simply by running the following script:
As you can see, the configuration file is not changed except the stage type.
At this point, the training strategy used comes from the qat_trainer and calibration_trainer in the config file.
By setting model_convert_pipeline when training quantized models, the corresponding floating-point model can be converted into a quantized model, as below:
For key steps in quantization training, e.g., preparing the floating-point model, operator substitution, inserting quantization and inverse quantization nodes, setting quantization parameters, and operator fusion, etc.,
please read the Quantized Awareness Training (QAT) section.
As previously mentioned, the quantization training is actually the finetuning based on the pure floating-point training.
Therefore, in the quantization training, set the initial learning rate is to one-tenth of the floating-point training, and the number of epochs of the training will greatly decrease as well.
The most important thing is that when defining model, we need to set pretrained to the address of the trained pure floating-point model.
After these simple adjustments, we can start training our quantized model.
