Multiple input tensorflow

AI Discussions
1

I have problem with data preparation with Tensorflow: This is my code

class Data:
    data = []
    init = False
    datagen = ImageDataGenerator(rescale=1./255.)
    #initize
    def __init__(self, path, img_size = (640, 640)):
        all_file = os.listdir(path) #take all couple files
        #load couple images
        data1 = []
        data2 = []
        label = []
        for i in all_file:
            #take couple path
            if platform.system() == 'Darwin' and i.startswith('.'):
                continue
            temp_path = os.listdir(path + '/' + i)
            temp_path.pop(temp_path.index('label.txt'))
            f = open(path +'/' + i + '/label.txt', "r")
            label.append(int(f.read()))
            data1.append(cv2.resize(cv2.imread(path +'/' + i + '/' + temp_path[0]),img_size))
            data2.append(cv2.resize(cv2.imread(path +'/' + i + '/' + temp_path[1]),img_size))
            
        self.data = np.array([data1, data2])
        self.label = np.array(label)
        self.init = True

    def load_data_generator(self, b_size):
        if not self.init :
            raise Exception('Data need to be initialized first')
        # print(np.shape(self.data))
        # generator = self.datagen.flow(x = part_data,y = part_label, batch_size=8)
        
        genX1 = self.datagen.flow(x = self.data[0],
                                  y = self.label,
                                batch_size = b_size,
                                shuffle=False, 
                                seed=7)
    
        genX2 = self.datagen.flow(x = self.data[1],
                                  y = self.label,
                                batch_size = b_size,
                                shuffle=False, 
                                seed=7)
        while True:
            X1i = genX1.next()
            X2i = genX2.next()
            yield (X1i[0], X2i[0]), X2i[1]
        

data = Data('/Users/admin/Desktop/Long journey/Research/Test')
#Test model
from tensorflow.keras.applications.resnet import ResNet101
from tensorflow.keras.layers import concatenate, Dense
from tensorflow.keras import layers, Model
import tensorflow as tf

resnet_1 = ResNet101(input_shape = (320, 320, 3), 
                                include_top = False, 
                                weights = None)
resnet_2 = ResNet101(input_shape = (320, 320, 3), 
                                include_top = False, 
                                weights = None)
x = resnet_1.layers[-2].output
y = resnet_2.layers[-2].output
#fix duplicate name
for layer in resnet_1.layers :
    layer._name = layer.name + str('_1')
for layer in resnet_2.layers :
    layer._name = layer.name + str('_2')
# combine the output of the two branches
combined = concatenate([x, y])
# apply a FC layer and then a regression prediction on the
# combined outputs
z = Dense(4096, activation="relu")(combined)
z = Dense(1, activation="sigmoid")(z)
# our model will accept the inputs of the two branches and
# then output a single value
model = Model(inputs=[resnet_1.input, resnet_2.input], outputs=z)
model.compile(loss=tf.keras.losses.BinaryCrossentropy(), optimizer='adam')

tmp = next(data.load_data_generator(8))
print(np.shape(a))
print(np.shape(b))
model.fit_generator(tmp,

and it throw this error:

Epoch 1/2
Traceback (most recent call last):
  File "/Users/admin/Desktop/Long journey/Research/Model/Data/input.py", line 93, in <module>
    model.fit(data.load_data_generator(8),
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/var/folders/x6/4mrrj52939970dsbyznz2qwr0000gn/T/__autograph_generated_fileoo26010v.py", line 15, in tf__train_function
    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
ValueError: in user code:

    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/training.py", line 1338, in train_function  *
        return step_function(self, iterator)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/training.py", line 1322, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/training.py", line 1303, in run_step  **
        outputs = model.train_step(data)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/training.py", line 1081, in train_step
        loss = self.compute_loss(x, y, y_pred, sample_weight)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/training.py", line 1139, in compute_loss
        return self.compiled_loss(
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/engine/compile_utils.py", line 265, in __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/losses.py", line 142, in __call__
        losses = call_fn(y_true, y_pred)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/losses.py", line 268, in call  **
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/losses.py", line 2432, in binary_crossentropy
        backend.binary_crossentropy(y_true, y_pred, from_logits=from_logits),
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/backend.py", line 5809, in binary_crossentropy
        return tf.nn.sigmoid_cross_entropy_with_logits(

    ValueError: `logits` and `labels` must have the same shape, received ((None, None, None, 1) vs (None,)).

I don’t have experience with config double input in tensorflow, what should I do? I hope I can recieve a sample project or advice for this thing, I am stucking in this think too long

2

Activation of the final tensorflow layer is sigmoid. So, there’s no need to worry about logits.
Please use model.summary() to understand the output shapes of each layer and confirm if it matches your problem expectation.

It’d be helpful if you provided more information about the course this topic belogs to.
Here’s the community user guide to get started.

3

Actually I learnt about tranfer learning at Coursera. However, this is my personal project relates to this knowledge. Beside that, this problem in this project is data preparation with ImageDataGenerator when input is twice.

Model belike:

           x1                               x2
        backbone                         backbone
             --------- concatenation --------
                            |
                   Fully Conected Layer
                            |
                          output

But I don’t know how to prepare image data for model like this

4

When you say coursera, are you referring to deep learning specialization? If so, please see week 4 assignment 1.

5

Hmmm. I learnt AI TensorFlow Developer Professional Certificate, but problem is my personal project. I already comprehend and complete this lesson but I meet new problem using ImageDataGenerator for model have double image input. I don’t know how to prepare data for model input 2 image simultaneously (You can see my structure model above). You can see the error received ((None, None, None, 1) vs (None,)) because my load_data_generator function have problem and I don’t know how to fix it. I stuck at this thing more than 1 weeks, it’s really hard :smiling_face_with_tear:

6

I recommend you take the mentioned specialization for more theory on deep learning.
You can use a layer like Flatten layer where appropriate to reshape the Conv 3 dimensional output to a 1 dimensional output.