#Qconfig Configuration

#Definition of QConfig

qconfig (Quantization Configuration) refers to the set of key parameters for quantization, which determines the quantization method of a deep learning model. Before preparing QAT/Calibration models, the qconfig must be set for the model.

For detailed principles of qconfig, readers can refer to the section Building QConfig.

#Configuring qconfig Using QconfigSetter (Recommended)

In large models or complex networks, manually setting QConfig for each layer is very tedious.
QconfigSetter, combined with templates, can automatically distribute configurations based on the computation graph.
QconfigSetter relies on the computation graph (Graph Mode) generated during the prepare stage and applies rules sequentially according to the template list.
The core structure of QconfigSetter is as follows:

QconfigSetter(
    reference_qconfig=...,  # 1. Provide the base qconfig for `observer`
    templates=[...],        # 2. Core: List of configuration rules (applied in order)
    enable_optimize=True,   # 3. If True, automatically enable graph optimization
    save_dir=...,           # 4. Path to save qconfig results (qconfig.pt file) and changelog
    custom_qconfig_mapping=... # 5. Use a dictionary to specify custom qconfig for individual operators
)

#Setting reference_qconfig

We recommend using get_qconfig to quickly configure reference_qconfig. An example is:

import torch
from horizon_plugin_pytorch.quantization import get_qconfig
from horizon_plugin_pytorch.quantization.observer_v2 import MinMaxObserver
from horizon_plugin_pytorch.quantization.qconfig import QConfig
from horizon_plugin_pytorch.quantization.fake_quantize import FakeQuantize
from horizon_plugin_pytorch.dtype import qint8
reference_qconfig = get_qconfig(
    observer=MinMaxObserver,   # Type of input/output observer. 
    in_dtype=None,             # Input data type. Set based on operator support. 
                               # None means the `input` keyword in QConfig is None. Default is None.
    weight_dtype=qint8,        # Weight data type. Set based on operator support. 
                               # None means the `weight` keyword in QConfig is None. Default is qint8.
    out_dtype=qint8,           # Output data type. Set based on operator support. 
                               # None means the `output` keyword in QConfig is None. Default is qint8.
    fix_scale=True,            # Whether to fix the input/output scale.
)

#qconfig Configuration Templates (Templates)

Templates are applied in the order they are listed, with later-defined templates overriding earlier ones (or specific names overriding global settings). For more detailed explanations, refer to the Building QConfig section.

Common templates include:

1. ModuleNameTemplate (General): Sets dtype based on operator names or prefixes.
   "": dtype represents the global default configuration. Longer names (more specific) have higher priority. Supports setting fixed thresholds.

2. ConvDtypeTemplate: Specifically used for batch configuration of the input and weight types for convolution layers.

3. MatmulDtypeTemplate: Specifically used for batch configuration of the input types for matrix multiplication (Linear/MatMul).

4. SensitivityTemplate: Automatically sets the Top-N sensitive layers to high precision based on sensitivity analysis results.

5. LoadFromFileTemplate: Loads a qconfig.pt file to reproduce previous quantization configurations. In this case, enable_optimize must be set to False, otherwise the correctness of the configuration results cannot be guaranteed, and there may be CPU operators during deployment.

An example of a commonly used template configuration is:

``` python 
import torch
import copy
import os
from horizon_plugin_pytorch.dtype import qint8, qint16
from horizon_plugin_pytorch.quantization import get_qconfig, observer_v2, prepare, PrepareMethod
from horizon_plugin_pytorch.quantization.qconfig_setter import (
    ConvDtypeTemplate,
    MatmulDtypeTemplate,
    ModuleNameTemplate,
    QconfigSetter,
)

# 1. Defining Configuration Templates (Templates)
# The order here is very important:
# First, set the global default to Float16, then override it with specific Conv/Matmul configurations.
q_template = [ 
    # [Strategy 1] Global fallback: All operators default to output Float16 (simulated pseudo-conversion)
    ModuleNameTemplate({"": torch.float16}),
    
    # [Strategy 2] MatMul: Force all MatMul operator inputs to Int16
    MatmulDtypeTemplate(  
        input_dtypes=[qint16, qint16],
    ),
    
    # [Strategy 3] Conv: Input Int16 (to preserve precision), Weight Int8
    ConvDtypeTemplate( 
        input_dtype=qint16,
        weight_dtype=qint8, 
    ),
]

#Configuring Custom qconfig

Although qconfig templates can cover most model qconfig configuration needs, QconfigSetter also provides an interface to configure qconfig for individual operators in the model.

First, users can directly configure a qconfig. A qconfig object can set the input, weight, and output keywords, which represent the quantization configuration for the operator's input, weight, and output, respectively.
When preparing the model, these configurations determine whether to insert FakeQuantize or FakeCast nodes at the corresponding positions. None means no nodes will be inserted.

FakeQuantize is a pseudo-quantization node that performs quantization and dequantization operations on the input. Inserting pseudo-quantization simulates the quantization error during the forward pass of the floating-point model. It is mainly used for quantizing fixed-point operators, such as qint8 and qint16.

FakeCast is a pseudo-conversion node that converts the input to float32. If the data type is float16, it also simulates truncation errors caused by converting to float16. This node is mainly used to mark operators that require floating-point computation.

An example of a Qconfig definition is:

import torch
from horizon_plugin_pytorch.quantization.qconfig import QConfig
from horizon_plugin_pytorch.quantization.fake_quantize import FakeQuantize
from horizon_plugin_pytorch.quantization.fake_cast import FakeCast
from horizon_plugin_pytorch.quantization.observer_v2 import MinMaxObserver
from horizon_plugin_pytorch.dtype import qint8

```python
qconfig = QConfig(
    input=None,
    weight=FakeQuantize.with_args(
        observer=MinMaxObserver,
        dtype=qint8,
        qscheme=torch.per_channel_symmetric,
        ch_axis=0,
    ),
    output=FakeCast.with_args(dtype=torch.float16),
    # activation=xxx Early usage, equivalent to the `output` keyword. 
    # It is still supported for compatibility, but using the `output` keyword is recommended.
)

This qconfig defines the weight type as per-channel symmetric quantization, with the type qint8 and the observer as MinMaxObserver.

With the qconfig, users can use custom_qconfig_mapping to specify qconfig for individual operators in the model:

qconfig_mapping = {
    "module_name": qconfig
}
QconfigSetter(
    reference_qconfig=..., 
    templates=[...],       
    enable_optimize=True,  
    save_dir=...,           
    custom_qconfig_mapping=qconfig_mapping
)

This allows users to have complete control over the qconfig of the model.