视觉变压器注意图由关键点位置 - 张量

发布于 2025-01-24 20:29:53 字数 867 浏览 2 评论 0 原文

我已经在TensorFlow上培训了一个VIT模型，以基于 https://github.com/yangsenius/yangsenius/yangsenius/transpose 我想模拟每个关键点的关注图： https://raw.githubusercontent.com/yangsenius/transpose/main/main/attention_map_image_image_image_depperency_transpose_transpose_0.00075.jpg

我在Pytorch上找到了代码

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

我不咬妳我踢妳 2025-01-31 20:29:53

我通过获取多头注意层的上一层的输出并通过多头注意来解决它：

atten_maps_hooks = [Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_0') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_1') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_2') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_3') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_4') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_5') - 1].output)]

for i in range(len(atten_maps_hooks)):
      temp = atten_maps_hooks[i].predict(input)
      mha, scores = model.get_layer('encoded_' + str(i))(temp, temp, return_attention_scores = True)
      enc_atten_maps_hwhw.append(scores.numpy()[0].reshape(shape + shape))

I have solved it by getting the output of the previous layer of the multihead attention layer and passing it by the multihead attention:

atten_maps_hooks = [Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_0') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_1') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_2') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_3') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_4') - 1].output),
                        Model(inputs = model.input, outputs = model.layers[getLayerIndexByName(model, 'encoded_5') - 1].output)]

for i in range(len(atten_maps_hooks)):
      temp = atten_maps_hooks[i].predict(input)
      mha, scores = model.get_layer('encoded_' + str(i))(temp, temp, return_attention_scores = True)
      enc_atten_maps_hwhw.append(scores.numpy()[0].reshape(shape + shape))

回复收藏 0 原文

~没有更多了~