TypeError：添加的层必须是类层的实例。找到：张量（“连接_6/concat:0”，shape=（无，4608），dtype=float32）

Question

我试图实现一个VQA模型，在这个模型中，我结合了一个图像和一个语言模型。我的模型定义是：

def VQA_MODEL():
    image_feature_size          = 4096
    word_feature_size           = 300
    number_of_LSTM              = 3
    number_of_hidden_units_LSTM = 512
    max_length_questions        = 30
    number_of_dense_layers      = 3
    number_of_hidden_units      = 1024
    activation_function         = 'tanh'
    dropout_pct                 = 0.5

    # Image model
    model_image = Sequential()
    model_image.add(Reshape((image_feature_size,), input_shape=(image_feature_size,)))

   # Language Model
   model_language = Sequential()
   model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=True,input_shape=(max_length_questions, word_feature_size)))
   model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=True))  
   model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=False))


   # combined model
   model = Sequential()
   model.add(concatenate([model_language.output, model_image.output]))

   for _ in range(number_of_dense_layers):
       model.add(Dense(number_of_hidden_units, kernel_initializer='uniform', activation= activation_function))
       model.add(Dropout(dropout_pct))

   model.add(Dense(50, activation='softmax'))

   return model

model = VQA_MODEL()
model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
model.fit(train_X, train_Y, batch_size = batch_size, epochs=nb_epoch)

我似乎不明白如何解决这个错误。任何线索将不胜感激

Answer 1

这个错误是因为带有小写字母的concatenate不是一个layer，只有带有大写字母的c是一个层。然而，这在你的情况下也不起作用

由于您的组合模型不是sequential，并且使用来自两个并行或不同模型的输入，因此最好使用FunctionalAPI。以下代码应该可以工作：

from tensorflow.keras import Sequential, Model
from tensorflow.keras.layers import LSTM, Reshape, Dense, Dropout, concatenate, Concatenate
def VQA_MODEL():
    image_feature_size          = 4096
    word_feature_size           = 300
    number_of_LSTM              = 3
    number_of_hidden_units_LSTM = 512
    max_length_questions        = 30
    number_of_dense_layers      = 3
    number_of_hidden_units      = 1024
    activation_function         = 'tanh'
    dropout_pct                 = 0.5

    # Image model
    model_image = Sequential()
    model_image.add(Reshape((image_feature_size,), input_shape=(image_feature_size,)))

   # Language Model
    model_language = Sequential()
    model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=True,input_shape=(max_length_questions, word_feature_size)))
    model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=True))  
    model_language.add(LSTM(number_of_hidden_units_LSTM, return_sequences=False))


    # combined model
    x = Concatenate()([model_language.output, model_image.output])

    for _ in range(number_of_dense_layers):
        x = Dense(number_of_hidden_units, kernel_initializer='uniform', activation= activation_function)(x)
        x = Dropout(dropout_pct)(x)
    x = Dense(50, activation='softmax')(x)
    model = Model(inputs = [model_language.input, model_image.input], outputs=x)
    return model

model = VQA_MODEL()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) 
model.fit(train_X, train_Y, batch_size = batch_size, epochs=nb_epoch)