Grad-CAM 迁移学习错误：尝试将具有不支持类型（<类 “nonetype”>）的值（无）转换为张量

Question

在使用转移学习技术训练机器学习模型之后，我正在尝试运行Grad-CAM，以对某些图像进行分类。 我使用的模型是Mobilenetv2。我在其中添加了一个分类器部分。我同时完成了模型培训和微调零件，并且该模型本身效果很好。

但是，当我尝试运行Grad-CAM以生成热图时，它给了我一个错误消息，如下所示：

valueError：尝试使用不支持的类型（＆lt;类'nontype'＆gt;类型转换值（无）； ）张量。

这是模型摘要：

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 200, 320, 3)]     0         
                                                                 
 mobilenetv2_1.00_224 (Funct  (None, 7, 10, 1280)      2257984   
 ional)                                                          
                                                                 
 flatten (Flatten)           (None, 89600)             0         
                                                                 
 dense (Dense)               (None, 6)                 537606    
                                                                 
=================================================================
Total params: 2,795,590
Trainable params: 537,606
Non-trainable params: 2,257,984
_________________________________________________________________
None

我使用以下代码查看MobilenetV2的层，并在Grad-Cam函数中需要提及最后一个卷积层的名称：

model.get_layer('mobilenetv2_1.00_224').summary()

def make_gradcam_heatmap(img_array, model, base_model_name,last_conv_layer_name):
    # First, we create a model that maps the input image to the activations
    # of the last conv layer as well as the output predictions
    grad_model = tf.keras.models.Model(
        [model.inputs], [model.get_layer(base_model_name).get_layer(last_conv_layer_name).output, model.output]
    )

    # Then, we compute the gradient of the top predicted class for our input image
    # with respect to the activations of the last conv layer
    with tf.GradientTape() as tape:
        last_conv_layer_output, preds = grad_model(img_array)
        class_channel = preds[:, 0]
    #print('preds: ',preds)    
    #print('last conv layer output:\n',last_conv_layer_output)
    #print('class channel:\n',class_channel)
    
    # This is the gradient of the output neuron (top predicted or chosen)
    # with regard to the output feature map of the last conv layer
    grads = tape.gradient(class_channel, last_conv_layer_output)
    #print('grads:\n',grads)

    # This is a vector where each entry is the mean intensity of the gradient
    # over a specific feature map channel
    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))

    # We multiply each channel in the feature map array
    # by "how important this channel is" with regard to the top predicted class
    # then sum all the channels to obtain the heatmap class activation
    last_conv_layer_output = last_conv_layer_output[0]
    heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]
    heatmap = tf.squeeze(heatmap)

    # For visualization purpose, we will also normalize the heatmap between 0 & 1
    heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)
    heatmap = heatmap.numpy()
    heatmap_resized = resize(heatmap,(200, 320))
    return heatmap_resized

model.layers[-1].activation = None


plt.figure(figsize = (6,12),dpi = 450)

for ii in range(6):
    # Generate class activation heatmap
    heatmap = make_gradcam_heatmap(X_test_pp[ii][np.newaxis,:,:], model, "mobilenetv2_1.00_224","Conv_1")
    plt.subplot(6,4,ii+1)
    plt.imshow(X_test_pp[ii])
    plt.title('Label: '+class_names[Y_test[ii]]+'| Pred: '+class_names[np.argmax(Y_pred,axis=1)[ii]],fontdict = {'fontsize' : 4})
    plt.imshow(heatmap, alpha = 0.35, cmap = "inferno")
    #plt.colorbar()
    plt.axis("off")
plt.show()

详细信息错误消息：

ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_5016/4168142392.py in <module>
      8 for ii in range(6):
      9     # Generate class activation heatmap
---> 10     heatmap = make_gradcam_heatmap(X_test_pp[ii][np.newaxis,:,:], model, "mobilenetv2_1.00_224","Conv_1")
     11     plt.subplot(6,4,ii+1)
     12     plt.imshow(X_test_pp[ii])

~\AppData\Local\Temp/ipykernel_5016/3660858513.py in make_gradcam_heatmap(img_array, model, base_model_name, last_conv_layer_name)
     22     # This is a vector where each entry is the mean intensity of the gradient
     23     # over a specific feature map channel
---> 24     pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
     25 
     26     # We multiply each channel in the feature map array

C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\util\traceback_utils.py in error_handler(*args, **kwargs)
    151     except Exception as e:
    152       filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153       raise e.with_traceback(filtered_tb) from None
    154     finally:
    155       del filtered_tb

C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
    100       dtype = dtypes.as_dtype(dtype).as_datatype_enum
    101   ctx.ensure_initialized()
--> 102   return ops.EagerTensor(value, ctx.device_name, dtype)
    103 
    104 

ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.

<Figure size 2700x5400 with 0 Axes>

这是我的问题， 首先，我给Grad-CAM的最后一个卷积层的名称正确吗？如果没有，那应该是什么？ 此错误消息的原因是什么，因为当我自己定义整个模型结构时，它正在工作？

到目前为止，我已经做过的事情：

我尝试调试代码，以查看此错误消息背后的原因是什么。 我意识到以下行返回“无”作为梯度，但是我不确定为什么呢？以及我如何解决这个问题。

grads = tape.gradient（class_channel，last_conv_layer_output）

我还尝试了最后一个卷积层的不同层名称，但它也不起作用。

我稍微读到了一些有关此问题的信息，并注意到有些人提到所有库中都需要直接从Tensorflow导入。这是我进口的内容：

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.model_selection import train_test_split
import tensorflow_datasets as tfds 
import glob
from skimage import transform
import cv2
import seaborn as sns
from skimage.transform import resize
from sklearn.metrics import confusion_matrix
from tensorflow.keras import backend as K

谢谢大家。

Answer 1

您的整个CNN型号保存在一层中。 mobilenetv2_1.00_224 - CNN模型

model = model.layers[1]

在加载模型时指定输入形状，以避免将模型保存在单层中，例如这样。

input = Input(shape=(640, 640, 3))
model = VGG16(include_top=False,input_tensor=input)