Mobilenet：Gradcam的转移学习

Question

我对所有这一切都是新手，所以请对这个问题友好:)

我想做的就是使用转移学习技术训练Mobilenet分类器，然后实施GARGCAM技术，以了解我的模型正在研究。

1. 我创建了一个模型

  input_layer = tf.keras.layers.input（shape = img_shape）
x = preprocess_input（input_layer）
y = base_model（x）
y = tf.keras.layers.globalaveratepooling2d（）（y）
y = tf.keras.layers.dropout（0.2）（y）
输出= tf.keras.layers.dense（5）（y）
模型= tf.keras.model（inputs = input_layer，outputs = outputs）
model.summary（）
 

模型摘要：

Model: "functional_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         [(None, 224, 224, 3)]     0         
_________________________________________________________________
tf_op_layer_RealDiv_1 (Tenso [(None, 224, 224, 3)]     0         
_________________________________________________________________
tf_op_layer_Sub_1 (TensorFlo [(None, 224, 224, 3)]     0         
_________________________________________________________________
mobilenetv2_1.00_224 (Functi (None, 7, 7, 1280)        2257984   
_________________________________________________________________
global_average_pooling2d_1 ( (None, 1280)              0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 1280)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 5)                 6405      
=================================================================
Total params: 2,264,389
Trainable params: 6,405
Non-trainable params: 2,257,984
_________________________________________________________________

2. 将其传递给Grad Cam算法，但是Grad Cam算法无法找到最后的卷积层

合理的解决方案： 如果我没有在模型中添加未包装的Mobilenet层，而不是拥有封装的“ Mobilenetv2_1.00_224

层

' 我可以将数据增强和PRE_PROCESSING层添加到Mobilenet Un -Inalper层的模型。

提前

感谢 ankit

Answer 1

@skruff看看这是否有帮助

def make_gradcam_heatmap(img_array, model, last_conv_layer_name, pred_index=None):
# First, we create a model that maps the input image to the activations
# of the last conv layer as well as the output predictions
grad_model = tf.keras.models.Model(
    [model.inputs], [model.get_layer(last_conv_layer_name).output, model.output]
)

# Then, we compute the gradient of the top predicted class for our input image
# with respect to the activations of the last conv layer
with tf.GradientTape() as tape:
    last_conv_layer_output, preds = grad_model(img_array)
    if pred_index is None:
        pred_index = tf.argmax(preds[0])
    class_channel = preds[:, pred_index]

# This is the gradient of the output neuron (top predicted or chosen)
# with regard to the output feature map of the last conv layer
grads = tape.gradient(class_channel, last_conv_layer_output)

# This is a vector where each entry is the mean intensity of the gradient
# over a specific feature map channel
pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))

# We multiply each channel in the feature map array
# by "how important this channel is" with regard to the top predicted class
# then sum all the channels to obtain the heatmap class activation
last_conv_layer_output = last_conv_layer_output[0]
heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap)

# For visualization purpose, we will also normalize the heatmap between 0 & 1
heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)
return heatmap.numpy()