#Writing Specifications of set_qconfig and Customization of qconfig

#Writing Specifications of set_qconfig

When defining the model to be quantified, the model set_qconfig method needs to be implemented to configure the quantization method.

The current QConfig interface is provided by hat.utils.qconfig_manager. Call hat.utils.qconfig_manager in set_qconfig to implement the setting of the module Qconfig, e.g.:

# Note: This sample code illustrates the rules for implementing the set_qconfig method and is not the full quantitative model code.

class Head(nn.Module):
    def __init__(self):
        super(Head, self).__init__()
        self.out_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=1)

    def forward(self):
        ...
        
    def set_qconfig(self):
        # If the network final output layer is conv, you can set it to out_qconfig separately to get a more accurate output
        from hat.utils import qconfig_manager
        self.out_conv.qconfig = qconfig_manager.get_default_qat_out_qconfig()

class Backbone(nn.Module):
    def __init__(self):
        super(Backbone, self).__init__()
        self.conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=1)
        
    def forward(self):
        ...

    # When there is no special layer in backbone and no layer needs to set QConfig=None, 
    # i.e., when setting default_qat_qconfig is needed, you can leave out the set_qconfig() method 
    # def set_qconfig(self):

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.backbone = Backbone()
        self.head = Head()
        self.loss = nn.CrossEntropyLoss()
    
    def forward(self):
        ...
        
    # Need to implement set_qconfig method for the parent module
    def set_qconfig(self):
        from hat.utils import qconfig_manager
        # 1. first specify the qconfig of the parent module,
        # if qconfig is not set for the child module, the child module will
        # automatically use the qconfig of the parent module
        self.qconfig = qconfig_manager.get_default_qat_qconfig()
        
        # 2. If a submodule has a special layer and implemented the set_qconfig method, call
        if self.head is not None:
            if hasattr(self.head, "set_qconfig"):
                self.head.set_qconfig()
                
        # 3. If there is a submodule that does not need to set Qconfig, 
        # you need to set Qconfig to None
        if self.loss is not None:
            self.loss.qconfig = None

#Customize QAT QConfig Parameters

Using custom QConfig in QAT training is supported in HAT, simply configure the qconfig_params parameter in the qat_solver of the config file:

qat_solver = dict(
    trainer=qat_trainer,
    quantize=True,
    ...
    qconfig_params=dict(
        dtype="qint8",
        activation_fake_quant="fake_quant",
        weight_fake_quant="fake_quant",
        activation_qkwargs=dict(
            averaging_constant=0,
        ),
        weight_qkwargs=dict(
            averaging_constant=1,
        ),
    ),
    ...
)

qconfig_params has five main parameter configuration items: dtype, activation_fake_quant, weight_fake_quant, activation_qkwargs and weight_qkwargs.

• dtype: dtype is the quantization bit type, supporting "qint8" (default).

• activation_fake_quant: Quantifier for activation, supporting "fake_quant" (default), "lsq", and "pact".

• weight_fake_quant: Quantifier for weight. Supporting "fake_quant" (default), "lsq", and "pact".

• activation_qkwargs: Parameters of activation quantifier.

  • When activation_fake_quant is "fake_quant", activation_qkwargs can be set as below:

  activation_qkwargs=dict(
      observer=MovingAverageMinMaxObserver,   # Specifies observer. In general, default can be used.
      averaging_constant=0.01,                # Sets scale update factor 
  )
  • When activation_fake_quant is "lsq", activation_qkwargs can be set as below:

  activation_qkwargs=dict(
      observer=MovingAverageMinMaxObserver,   # Specifies observer. In general, default can be used.
      scale=1.0,                              # Specifies the initial scale. In general, default can be used.
      zero_point=0.0,                         # Specifies the initial zero_point. In general, default can be used.
      use_grad_scaling=False,                 # Specifies whether the gradients of scale and zero_point are normalized
                                              # by a constant. False by default. In general, default can be used.
  )
  • When activation_fake_quant is "pact", activation_qkwargs can be set as below:

  activation_qkwargs=dict(
      observer=MovingAverageMinMaxObserver,   # Specifies observer. In general, default can be used.
      alpha=6.0,                              # Specifies the clip parameter of activation. The default value is 6.0. 
                                              # In general, default can be used.
  )

• weight_qkwargs: Specifies the parameters for the weight quantifier. Except that the default observer for weight_qkwargs is MovingAveragePerChannelMinMaxObserver, other parameters and usage are the same as activation_qkwargs.