• English
  • 3.2.0

    • Eager Mode

    The horizon_plugin_pytorch currently supports quantization using the eager mode, however, we no longer recommend using this mode. For quantization in eager mode, currently you are still supported by our horizon_plugin_pytorch. The overall flow of the Eager mode is also based on PyTorch's quantization interface and ideas, so we recommend that you read the Eager mode section of the PyTorch official document.

    Difference from fx mode

    The main differences between using eager mode and fx mode in horizon_plugin_pytorch are:

    • The eager mode only supports module operators. Before carryout model quantization, you need to manually replace the functional operators in the floating-point model with Module-type operators in PyTorch or proprietary operators defined in horizon_plugin_pytorch, including but not limited to:
    Original floating-point operatorOperator to be replaced
    torch.nn.functional.relutorch.nn.ReLU()
    a + b /
    torch.add    		horizon.nn.quantized.FloatFunctional().add
    Tensor.exphorizon.nn.Exp()
    torch.nn.functional.interpolatehorizon.nn.Interpolate()
    • You must manually define the operators to be fused and explicitly call the fusion function before carryout model quantization, and also specify the use of the fuser_func provided in horizon_plugin_pytorch when calling it. As shown below:
    import torch
from torch import nn
import horizon_plugin_pytorch as horizon


class ConvBNReLU(nn.Sequential):
    def __init__(self, in_channels, out_channels, kernel_size):
        super(ConvBNReLU, self).__init__(
            nn.Conv2d(
            in_channels=in_channels,
            out_channels=out_channels,
            kernel_size=kernel_size
            ),
            nn.BatchNorm2d(num_features=out_channels),
            nn.ReLU()
        )

    # specify the operators that can be fuse
    def fuse_model(self):
        torch.quantization.fuse_modules(
            self,
            ['0', '1', '2'],
            inplace=True,
            # specify the fuse function provided in horizon_plugin_pytorch
            fuser_func=horizon.quantization.fuse_known_modules,
        )

float_model = ConvBNReLU(1, 1, 1)
# call the fuse function explicitly
float_model.fuse_model()

print(float_model)
# ConvBNReLU(
#   (0): ConvReLU2d(
#     (0): Conv2d(1, 1, kernel_size=(1, 1), stride=(1, 1))
#     (1): ReLU()
#   )
#   (1): Identity()
#   (2): Identity()
# )