预先训练的BERT不是LSTM层的正确形状：值误差，新数组的总大小必须不变

Question

我试图在暹罗神经网络上使用预训练的BERT模型。但是，我遇到了将BERT模型传递到共享LSTM层的问题。我遇到以下错误：

ValueError: Exception encountered when calling layer "reshape_4" (type Reshape).

total size of new array must be unchanged, input_shape = [768], output_shape = [64, 768, 1]

Call arguments received:
  • inputs=tf.Tensor(shape=(None, 768), dtype=float32)

我在其他几篇文章中阅读了我所馈入LSTM的尺寸应为[batch_size，768，1]。但是，当我尝试重塑时，我会遇到错误。如何解决此错误？

input_1 = Input(shape=(), dtype=tf.string, name='text')
preprocessed_text_1 = bert_preprocess(input_1)
outputs_1 = bert_encoder(preprocessed_text_1)
e1 = tf.keras.layers.Reshape((64, 768, 1))(outputs_1['pooled_output'])

input_2 = Input(shape=(), dtype=tf.string, name='text')
preprocessed_text_2 = bert_preprocess(input_2)
outputs_2 = bert_encoder(preprocessed_text_2)
e2 = Reshape((64, 768, 1))(outputs_2['pooled_output'])

lstm_layer = Bidirectional(LSTM(50, dropout=0.2, recurrent_dropout=0.2)) # Won't work on GPU

x1 = lstm_layer(e1)
x2 = lstm_layer(e2)

mhd = lambda x: exponent_neg_cosine_distance(x[0], x[1]) 
merged = Lambda(function=mhd, output_shape=lambda x: x[0], name='cosine_distance')([x1, x2])
preds = Dense(1, activation='sigmoid')(merged)
model = Model(inputs=[input_1, input_2], outputs=preds)

Answer 1

您必须从重塑层中删除批处理大小（= 64）。