加入2个keras层的输出

Question

我正在尝试使用KERAS实现联合模型，这是该模型的架构。

但是，我在子网和主网络的输入串联串联时很难。以下是我的代码：

import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Reshape, Concatenate


def Autoencoder():
  input = layers.Input(shape=(256, 256, 5))
  layers.Flatten()

  x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(input)
  x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same")(x)

  x = layers.Conv2D(64, (3, 3), activation="relu", padding="same")(x)
  x = layers.Conv2D(64, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same")(x)

  x = layers.Conv2D(128, (3, 3), activation="relu", padding="same")(x)
  x = layers.Conv2D(128, (3, 3), activation="relu", padding="same")(x)
  x = layers.MaxPooling2D((2, 2), padding="same", name='last_layer')(x)
  autoencoder = Model(input, x)
  return autoencoder.get_layer('last_layer')



def Subnetwork():
  input = layers.Input(shape=(12,1))

  x = layers.Flatten()(input)
  x = layers.Dense(4096, activation="relu")(x)
  x = layers.Reshape((32, 32, 4), name='last_layer')(x)
  subnetwork = Model(input, x)
  return subnetwork.get_layer('last_layer')

def Joint():
  layer_autoencoder = Autoencoder()
  layer_subnetwork = Subnetwork()
  merged= Concatenate([layer_autoencoder, layer_subnetwork])
  model = Model(inputs=[layer_autoencoder, layer_subnetwork], outputs=merged)
  return model

Model = Joint()
Model.summary()

错误消息看起来像这样：

ValueError: Found unexpected instance while processing input tensors for keras functional model. Expecting KerasTensor which is from tf.keras.Input() or output from keras layer call(). Got: <keras.layers.pooling.MaxPooling2D object at 0x7fbfd8634990>

有人知道是什么原因导致错误，什么是正确的解决方案？

Answer 1

您还应该在Joint模型中重新定义输入层，

def Joint():

  input_autoencoder = layers.Input(shape=(256, 256, 5))  ### define input layer
  layer_autoencoder = Autoencoder()(input_autoencoder)  ### pass input to Autoencoder
  input_subnetwork = layers.Input(shape=(12, 1))  ### define input layer
  layer_subnetwork = Subnetwork()(input_subnetwork)  ### pass input to Subnetwork
  
  merged= Concatenate()([layer_autoencoder, layer_subnetwork])  ### it's Concatenate()([...]) not Concatenate([...])
  model = Model([input_autoencoder, input_subnetwork], merged)  ### use correct inputs
  return model

还要注意Autoencoder和Subnetwork。它们必须返回 TF 模型实例。所以他们变成：

def Autoencoder():
  ...
  autoencoder = Model(input, x)
  return autoencoder

def Subnetwork():
  ...
  subnetwork = Model(input, x)
  return subnetwork