• English
  • 3.8.1

    • Predict Tool

    The tools/predict.py tool provided by the algorithm package serves as the core entry point for model inference. It supports single-GPU/multi-GPU and single-node/multi-node inference modes, and is compatible with the configuration and execution of different inference phases (e.g., floating-point inference, quantized model inference, fixed-point model inference). Flexible control over the entire inference pipeline can be achieved via concise command-line arguments.

    Usage

    Run tools/predict.py, which supports specifying the core training configuration through command-line parameters, and can also complete the definition of the refined training process by combining with the config file.

    Parameters Introduction

    ParameterDescription
    --stageDifferent training and inference stages. Valid choices: float, calibration, qat, int_infer.
    --config, -cPath to the config file.
    --device-ids, -idsList of GPUs to use.
    --dist-urlServer address for multi-machine operation, default is auto.
    --backendBackend used for multi-machine or multi-GPU communication.
    --ckptCheckpoint file for prediction.
    --launcherThe startup method for multi-machine operation. Please select "torch".
    --pipeline-testWhether to run pipeline test.

    Usgae Example

    Taking resnet50_imagenet as an example, to launch multi-GPU quantized model inference, For a detailed introduction to the configuration file, please refer to the [Configuration] section [config configuration] (../config/keywords).

    python3 tools/predict.py --stage qat \
                      --config configs/classification/resnet50_imagenet.py \
                      --device-ids 0,1

    For a detailed introduction to the configuration file, please refer to the [Configuration] section [config configuration] (../config/keywords).