重新调整多项式输出 - 张量
我们知道，MultiHeadateNtion的Keras API提供output_shape参数，您可以在其中指定需要投影到的输出的大小。但是，批处理大小和序列维度似乎并不可变。
例如：
layer = layers.MultiHeadAttention(num_heads=2, key_dim=2,output_shape=[5,])
target = tf.random.normal(shape=[3,5,1])
source = tf.random.normal(shape=[3,4,1])
output_tensor = layer(target, source)
此特定的调用将使我形成TensorShape（[3，5，5]）。我知道3的批次维度和序列维度是不可更改或可自定义的，因为查询键投影的工作方式。现在，我想使用另一个自定义层重塑此张量（我可以在外部进行，但我想将其保留为模型的一部分）。
为了重塑，我想使用自定义层，以[15,5]之类的东西，我尝试了类似的东西：
reshape = layers.Reshape((15,5))
然后应用Reshape层。但是，我无法重塑，因为Keras认为我试图重塑3,15,5，这是通过此错误证明的：“ reshape输入是一个具有75个值的张量，但是所请求的形状具有225 [OP：RESHAPE]“。
我还尝试了（-1,5），但是该操作根本不会更改我的张量，并且假设它已经正确。
是在外部自定义MHA输出的唯一方法，还是有任何方法可以通过模型的一层或一部分进行操作？
感谢帮助。
We know that MultiHeadAttention's Keras API offers an output_shape argument, where you can specify the size you need your output to be projected to. However, the batch size and the sequence dimension does not seem to be alterable.
For example:
layer = layers.MultiHeadAttention(num_heads=2, key_dim=2,output_shape=[5,])
target = tf.random.normal(shape=[3,5,1])
source = tf.random.normal(shape=[3,4,1])
output_tensor = layer(target, source)
This particular call will give me a shape of TensorShape([3, 5, 5]). I understand that the batch dimension of 3 and the sequence dimension are not alterable or customizable given how the query-key projection works. Now, I wanted to reshape this Tensor using another custom layer (I can do it externally, but I wanted to keep it as a part of the model).
For reshaping, I wanted to use a custom layer, to something like [15,5], I have tried something like this:
reshape = layers.Reshape((15,5))
And then applying the reshape layer. However, I am unable to reshape as Keras think I am trying to reshape to 3,15,5, which is proven by this error: "Input to reshape is a tensor with 75 values, but the requested shape has 225 [Op:Reshape]".
I have also tried (-1,5), but the operation does not change my tensor at all and assumes it is already correct.
Is the only way to customize the MHA output doing it externally, or is there any way I can do it by a layer or a part of a model?
Appreciate the help.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
重塑尝试的核心问题是您正在尝试弄平批处理维度。 KERAS中的所有标准层都将保持其恒定并重塑张量的其余部分。
例如，当您进行
reshape = layers.Reshape（（（15,5））时，实际上试图做的是重塑张量，同时保持批处理尺寸恒定。也就是说，给定Shape
的张量[3，5，5]（= 75elements），它试图将其变成[3， 15，5]（= 225元素 - 因此错误）然后您的下一个尝试，
reshape = layers.Reshape.Reshape（（ - 1,5）），试图重建形状对于3维张量（包括批量尺寸），您的张量已经进入。这就是为什么您看不到更改的原因。要处理批处理维度，您需要一个
lambda层。给出（例如
reshaped_out.hape），The core issue with your reshaping attempt is that you are trying to flatten the batch dimension. All of the standard layers in Keras would keep it constant and reshape rest of the tensor.
For example, when you do
reshape = layers.Reshape((15,5))you get,What your layer in fact trying to do is reshape the tensor while keeping the batch dimension constant. That is, given the tensor of shape
[3, 5, 5](=75elements) it is trying to get it into a shape of[3, 15, 5](=225elements - thus the error)Then your next attempt,
reshape = layers.Reshape((-1,5)), tries to recorrect shape to a 3 dimensional tensor (including batch dimension) which your tensor is already in. This is why you're not seeing a change.To meddle with the batch dimension, you'd need a
Lambdalayer.which gives (e.g.
reshaped_out.shape),