在TensorFlow后端使用Keras时，所有输入数组（x）的样本数应相同（再次询问）

Question

对不起，第一次不知道如何问一个好问题，所以请再问一次。
我创建了一个具有多个输入和输出的模型。模型的结构如下所示。
没有足够的声誉发布图像，因此图像为in this link

inputV = keras.layers.Input(shape=(4096,),name='inputV') #input for visible task
inputH = keras.layers.Input(shape=(4096,),name='inputH') #input for hidden task
hiddenTaskLayer = keras.layers.Dense(14,use_bias=False,activation='sigmoid',name='hiddenTask')
outputV = hiddenTaskLayer(inputV)
outputH = hiddenTaskLayer(inputH)
visibleTaskLayer = keras.layers.Dense(7,use_bias=False,activation='softmax',name='visibleTask')
outputV = visibleTaskLayer(outputV)
outputV = keras.layers.Activation('linear',name='outputV')(outputV)
outputH = keras.layers.Activation('linear',name='outputH')(outputH)
model = keras.models.Model(inputs=[inputH,inputV],outputs=[outputH,outputV])
model.compile(optimizer='sgd',
              loss={'outputH': 'msle', 'outputV': 'categorical_crossentropy'},
              loss_weights={'outputH': .5, 'outputV': 0.5})

Model: "model_2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
inputH (InputLayer)             [(None, 4096)]       0                                            
__________________________________________________________________________________________________
inputV (InputLayer)             [(None, 4096)]       0                                            
__________________________________________________________________________________________________
hiddenTask (Dense)              (None, 14)           57344       inputV[0][0]                     
                                                                 inputH[0][0]                     
__________________________________________________________________________________________________
visibleTask (Dense)             (None, 7)            98          hiddenTask[0][0]                 
__________________________________________________________________________________________________
outputH (Activation)            (None, 14)           0           hiddenTask[1][0]                 
__________________________________________________________________________________________________
outputV (Activation)            (None, 7)            0           visibleTask[0][0]                
==================================================================================================
Total params: 57,442
Trainable params: 57,442
Non-trainable params: 0
__________________________________________________________________________________________________

但是当我试着去适应这个模型的时候

model.fit({'inputH': XforHT, 'inputV': XforVT},
          {'outputH': YforHT, 'outputV': YforVT},
          epochs=10000, batch_size=100,
          shuffle=True,validation_split=0.33, 
          callbacks=[tensorboard_callback, cp_callback, scheduler_callback])

用我提供的数据集

print(XforHT.shape)
print(YforHT.shape)
print(XforVT.shape)
print(YforVT.shape)

(593, 4096)
(593, 14)
(327, 4096)
(327, 7)

错误提示为：

ValueError: All input arrays (x) should have the same number of samples. Got array shapes: [(593, 4096), (327, 4096)]

我知道如果将XforHT中的样本数减少到327，代码将运行良好，但我不想这样做。
对于不同的输入和输出，我确实有不同数量的样本，并且我确信每个输出都有相同数量的对应输入的样本。如果不放弃一些样品或寻找一些新样品，我可以做什么