Query about integrating yolov8 with my CNN model

I have successfully developed my CNN model but ,I need to integrate it with the yolov8 model for segmentation and real time detection. Can anyone please help me to do the integration

model = keras.Sequential()

model.add(data_augmentation)

model.add(layers.Resizing(64, 64))

model.add(layers.Conv2D(64, (3, 3), activation=‘relu’, input_shape=(64, 64, 3)))

model.add(layers.Conv2D(64, (3, 3), activation=‘relu’, input_shape=(64, 64, 3)))

model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(128, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.Conv2D(128, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(256, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.Conv2D(256, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(512, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.Conv2D(512, (3, 3), activation=‘relu’, padding=“same” ))

model.add(layers.BatchNormalization())

model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Flatten())

model.add(layers.Dense(2048, activation=‘relu’))

model.add(layers.Dense(2048, activation=‘relu’))

model.add(layers.Dropout(0.25))

model.add(layers.Dense(1024, activation=‘relu’))

model.add(layers.Dense(1024, activation=‘relu’))

model.add(layers.Dense(512, activation=‘relu’))

model.add(layers.Dense(512, activation=‘relu’))

model.add(layers.BatchNormalization())

model.add(layers.Dense(len(class_names), activation=‘softmax’))

model.compile(keras.optimizers.Adam(learning_rate=0.00005, beta_1=0.9, beta_2=0.999, amsgrad=True),

loss=tf.keras.losses.SparseCategoricalCrossentropy(),

metrics=[‘accuracy’, tf.keras.metrics.SparseTopKCategoricalAccuracy(k=2, name=“top_2”)])

this is my model summary and i need to integrate this model with yolo

Should we infer that your current architecture does not support real time detection, but the aspirational combined architecture will do? Sorry, but feel like I’m missing something here.

yea i need to integrate the yolo model for the segmentation but i didn’t find any solutions

Your model just looks like a multiclass classifier that does no localization of the objects within the images. YOLO does multiclass classification and object localization, so it seems that you could just use that. The only question is whether you need to do any additional training of YOLO on your particular data. In terms of the decision making here, it would also be helpful to know what your data looks like. E.g. does it contain different types of objects than YOLO is trained on? Not just stop signs and motorcycles and pedestrians, but also aardvarks and koalas? :nerd_face:

+1 to that

It also is doing only one call to softmax() per image, unlike YOLO, which can classify multiple objects (up to grid cell count ^2 * number of anchors) per input image.

I’m still not clear on whether real-time is needed or not, and if so, how combining the architectures would achieve that if this CNN cannot do it by its own self :man_shrugging:

In addition to what everyone else has said, why don’t you try fine-tuning yoloV8 on your data?

If you look at the docs they have some pre-trained models that you can use to start from.
Train - Ultralytics YOLOv8 Docs)-,%23%20load%20a%20pretrained%20model%20(recommended%20for%20training),-model%20%3D%20YOLO