在保留稀疏性和聚类的同时，将缩放层排除在张量量化中

发布于 2025-01-22 13:01:23 字数 3763 浏览 1 评论 0 原文

我正在关注 Stripped_clustered_model 。不幸的是，我的模型包含一个无法量化的层（重新缩放层）。为了解释这一点，我使用量化_annotate_layer 仅标记其他层进行量化。我是通过调用此代码来做到的：

def apply_quantization_to_non_rescaling(layer):
    if not isinstance(layer, tf.keras.layers.Rescaling):
        print('=> NOT Rescaling')
        return tfmot.quantization.keras.quantize_annotate_layer(layer, quantize_config=None)
    print('=> Rescaling')
    return layer

quant_aware_annotate_model = tf.keras.models.clone_model(
    stripped_clustered_model,
    clone_function=apply_quantization_to_non_rescaling,
)

pcqat_model = tfmot.quantization.keras.quantize_apply(
              quant_aware_annotate_model,
              tfmot.experimental.combine.Default8BitClusterPreserveQuantizeScheme(preserve_sparsity=True)
)

为了理解，我用 knatize_annotate_layer 标记了所有要量化的图层。稍后，我称量化实际执行此量化。但是，运行此代码会导致以下错误：

=> Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
Traceback (most recent call last):
  File "model_2.py", line 332, in <module>
    main()
  File "model_2.py", line 304, in main
    pcqat_model = tfmot.quantization.keras.quantize_apply(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/keras/metrics.py", line 74, in inner
    raise error
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/keras/metrics.py", line 69, in inner
    results = func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 474, in quantize_apply
    return keras.models.clone_model(
  File "/home/user/.local/lib/python3.8/site-packages/keras/models.py", line 453, in clone_model
    return _clone_sequential_model(
  File "/home/user/.local/lib/python3.8/site-packages/keras/models.py", line 330, in _clone_sequential_model
    if isinstance(layer, InputLayer) else layer_fn(layer))
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 408, in _quantize
    full_quantize_config = quantize_registry.get_quantize_config(layer)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/collaborative_optimizations/cluster_preserve/cluster_preserve_quantize_registry.py", line 293, in get_quantize_config
    quantize_config = (default_8bit_quantize_registry.
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/default_8bit/default_8bit_quantize_registry.py", line 272, in get_quantize_config
    raise ValueError(
ValueError: `get_quantize_config()` called on an unsupported layer <class 'keras.layers.preprocessing.image_preprocessing.Rescaling'>. Check if layer is supported by calling `supports()`. Alternatively, you can use `QuantizeConfig` to specify a behavior for your layer.

输出显示了所有内容，但第一层（即缩放层）被标记用于量化。但是，以下错误告诉我，还使用重新缩放层进行量化。

我如何从量化中排除重新缩放层？

更新22.4.2022： 无法选择回到默认量化策略。

pcqat_model = tfmot.quantization.keras.quantize_apply(
    quant_aware_annotate_model
)

通过使用而不是


pcqat_model = tfmot.quantization.keras.quantize_apply(
    quant_aware_annotate_model,
    tfmot.experimental.combine.Default8BitClusterPreserveQuantizeScheme(preserve_sparsity=True)
)

使用而不是保留稀疏性和聚类，因此

原文

I'm following this guide for performing quantization on my model stripped_clustered_model. Unfortunately, my model contains a layer, which can not be quantized (Rescaling layer). In order to account for that, I'm using quantize_annotate_layer to only mark the other layers for quantization. I'm doing that by calling this code:

def apply_quantization_to_non_rescaling(layer):
    if not isinstance(layer, tf.keras.layers.Rescaling):
        print('=> NOT Rescaling')
        return tfmot.quantization.keras.quantize_annotate_layer(layer, quantize_config=None)
    print('=> Rescaling')
    return layer

quant_aware_annotate_model = tf.keras.models.clone_model(
    stripped_clustered_model,
    clone_function=apply_quantization_to_non_rescaling,
)

pcqat_model = tfmot.quantization.keras.quantize_apply(
              quant_aware_annotate_model,
              tfmot.experimental.combine.Default8BitClusterPreserveQuantizeScheme(preserve_sparsity=True)
)

For my understanding, I mark all layers, which I want to quantize, with quantize_annotate_layer. Later, I call quantize_apply to actually perform this quantization. However, running this code leads to following error:

=> Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
=> NOT Rescaling
Traceback (most recent call last):
  File "model_2.py", line 332, in <module>
    main()
  File "model_2.py", line 304, in main
    pcqat_model = tfmot.quantization.keras.quantize_apply(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/keras/metrics.py", line 74, in inner
    raise error
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/keras/metrics.py", line 69, in inner
    results = func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 474, in quantize_apply
    return keras.models.clone_model(
  File "/home/user/.local/lib/python3.8/site-packages/keras/models.py", line 453, in clone_model
    return _clone_sequential_model(
  File "/home/user/.local/lib/python3.8/site-packages/keras/models.py", line 330, in _clone_sequential_model
    if isinstance(layer, InputLayer) else layer_fn(layer))
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 408, in _quantize
    full_quantize_config = quantize_registry.get_quantize_config(layer)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/collaborative_optimizations/cluster_preserve/cluster_preserve_quantize_registry.py", line 293, in get_quantize_config
    quantize_config = (default_8bit_quantize_registry.
  File "/usr/local/lib/python3.8/dist-packages/tensorflow_model_optimization/python/core/quantization/keras/default_8bit/default_8bit_quantize_registry.py", line 272, in get_quantize_config
    raise ValueError(
ValueError: `get_quantize_config()` called on an unsupported layer <class 'keras.layers.preprocessing.image_preprocessing.Rescaling'>. Check if layer is supported by calling `supports()`. Alternatively, you can use `QuantizeConfig` to specify a behavior for your layer.

The output shows me, that all, but the first layer (that is the rescaling layer) are marked for quantization. However, the following error tells me, that also the rescaling layer is used for quantization.

How can I exclude the rescaling layer from quantization?

Update 22.4.2022:
It is no option to fall back to the default quantization strategy by using

pcqat_model = tfmot.quantization.keras.quantize_apply(
    quant_aware_annotate_model
)

instead of


pcqat_model = tfmot.quantization.keras.quantize_apply(
    quant_aware_annotate_model,
    tfmot.experimental.combine.Default8BitClusterPreserveQuantizeScheme(preserve_sparsity=True)
)

since this would not preserve sparsity and clustering.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

只涨不跌 2025-01-29 13:01:23

因此，在传递另一个量化策略时，以某种方式出现了错误。如果您只使用 pcqat_model = tfmot.quantization.keras.quantize_apply（part_aware_annotate_model）它将起作用。我试图使用其他非实验量化策略，但它们也会引发一些错误。因此，如果您绝对设置了与默认策略相比，这无济于事，但是如果您只想使用量化，请使用默认情况。