MapTROE Training
The MapTROE reference algorithm is developed based on Horizon Algorithm Toolkit (HAT, Horizon's own deep learning algorithm toolkit). The training config for MapTROE is located under configs/map/ path. The following part takes configs/map/maptroe_henet_tinym_bevformer_nuscenes.py as an example to describe how to configure and train MapTROE model.
Training Process
If you just want to simply train the MapTROE model, then you can read this section first.
Similar to other tasks, HAT performs all training tasks and evaluation tasks in the form of tools + config.
After preparing the original dataset, take the following process to complete the whole training process.
Dataset Preparation
Here is an example of the nuscense dataset, which can be downloaded from https://www.nuscenes.org/nuscenes . Also, in order to improve the speed of training, we have done a packing of the original jpg format dataset to convert it to lmdb format. Just run the following script and it will be successful to achieve the conversion.
The above two commands correspond to transforming the training dataset and the validation dataset, respectively. In addition, the MapTROE model also requires standand definition map (SD map) for map fusion to improve model accuracy, which can be obtained from the Open Street Map(OSM) https://www.openstreetmap.org. After the packing is completed, the file structure in the data directory should look as follows.
The train_lmdb and val_lmdb are the training and validation datasets after packaging, and are the datasets that the network will eventually read. metas contains the map information used during model training and validation. osm contains the sd map information used in map fusion.
Model Training
Once datasets are ready, you can start training the floating-point FCOS-efficientnet detection network.
If you simply want to start such a training task, just run the following command:
The above two commands are respectively for the training of the floating-point model and the fixed-point model. The fixed-point model training needs to be based on the trained fixed-point model. For details on this, please read the Quantized Awareness Training (QAT) section.
Export the Fixed-point Model
Once you've completed your qat training, you can start exporting your quantized model. You can export it with the following command:
Model Validation
After completing the training, we get the trained float, calibration, and qat model. Similar to the training method, we can use the same method to complete metrics validation on the trained
model and get the metrics of Float, Calibration, and QAT respectively.
Similar to the model training, we can use --stage followed by float, calibration, or qat to validate the specific model respectively.
The following command can be used to verify the accuracy of a quantized model, but it should be noted that hbir must be exported first:
Model Inference and Results Visualization
HAT provides the infer_hbir.py script to visualize the inference results for the quantized model:
Fixed-point Model Checking and Compilation
The quantization training toolchain integrated in HAT is mainly prepared for the Horizon computing platform, so it is necessary to check and compile the quantized models. We provide a model checking interface in HAT, which can be used to check whether the defined quantization model can run normally on the BPU before training:
After the model is trained, you can use the compile_perf_hbir script to compile the quantized model into an HBM file that supports on-board running.
The above is the whole process from data preparation to the generation of quantized and deployable models.
Training Details
In this note, we explain some things that need to be considered for model training, mainly including settings related to config.
Model Construction
Where type under model indicates the name of the defined model, and the remaining variables indicate the other components of the model.
The advantage of defining the model this way is that we can easily replace the structure we want. For example, if we want to train a model with a backbone of resnet18, we just need to replace backbone under model .
Data Augmentation
Like the definition of model, the data augmentation process is implemented by defining two dicts data_loader and val_data_loader in the config file as follows, corresponding to training set and the processing flow of the validation set.
Where type directly uses the interface torch.utils.data.DataLoader that comes with pytorch, which represents the combination of batch_size size images together.
The only thing to be concerned about here is probably the dataset variable, data_path means the packed lmdb dataset path, map_path is the map information path, and sd_map_path is the sd map information path as we mentioned in the first part of the dataset preparation.
transforms contains a series of data augmentation. You can also achieve your own data augmentation by inserting a new dict in transforms.
Training Strategy
In order to train a model with high accuracy, a good training strategy is essential.
For each training task, the corresponding training strategy is defined in the config file as well, which can be seen from the variable float_trainer.
The float_trainer defines our training approach in the big picture, including the use of distributed_data_parallel_trainer, the number of epochs for model training, and the choice of optimizer.
Also, the callbacks reflect the small strategies used by the model during training and the operations that you want to implement, including the way to transform the learning rate (CosineAnnealingLrUpdater),
the metrics to validate the model during training (Validation), and the operations to save (Checkpoint) the model.
Of course, if you have operations that you want the model to implement during training, you can also add them in this dict way.
If reproducibility accuracy is needed, the training strategy in config is best not modified. Otherwise, unexpected training situations may occur.
Quantized Model Training
By using float_trainer, we can get a float model with high accuracy. Thus we can start to train the corresponding calibration and qat model. The corresponding training strategies are defined as follows.
Quantized training is in fact finetuning based on float training, so during quantization training, our qat learning rate is much smaller than the float learning rate.
The number of epochs for quantization training is largely reduced, most importantly, when defining the model , our pretrained needs to be set to the path of the previous model checkpoint.
The Settings of Qconfig
Before we begin quantization training, we need to set qconfig of the model. Float model will be converted into a quantized model by the settings of qconfig as follows.
cali_qconfig_setter and qat_qconfig_setter are qconfig settings for calibration model and qat model specifically. For qconfig settings, such as default templates and sensitivity templates, please read the
Quantized Awareness Training (QAT) Qconfig in Detail section.