Template matching in CNN

AI Discussions
1

I’m using tensorflow for my CNN model and tf.data for my preprocessing pipeline.
The problem I’m facing is as follows:

  • I have a template of size (224, 224) and I wish to do template matching on all the images (size - (256, 256)) in my training dataset.
  • For this purpose, I wish to use cv2.matchTemplate(image, template, method) function but for this to work, image must be a numpy array. In my case, image is a tensorflow tensor so I’m unable to work things out.

I’m writing the code that I’ve used below for reference:

def template_matching(image, labels):
    w, h = template.shape[::-1]
    method = eval('cv2.TM_CCOEFF')
    image_gray = tf.image.rgb_to_grayscale(image)
    res = cv2.matchTemplate(image_gray, template, method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
    top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)
    cropped_image = image[top_left[1]:bottom_right[1],top_left[0]:bottom_right[0],:]
    
    return cropped_image, labels

def parse_function(filename, label):
    image_string = tf.io.read_file(filename)
    image_decoded = tf.image.decode_jpeg(image_string, channels=CHANNELS)
    img = tf.image.resize(image_decoded, [IMG_SIZE, IMG_SIZE])
    
    return img, label

def dataset_augment(filenames, labels):    
    dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
    dataset = dataset.map(parse_function, num_parallel_calls=AUTOTUNE)
    dataset = dataset.map(template_matching, num_parallel_calls=AUTOTUNE)

    return dataset

What can be a possible solution to this problem?

2

Maybe I’m just missing your point, but it’s easy to convert back and forth between numpy arrays and TF tensors, right? Have you tried googling “how to convert a tf tensor to a numpy array”?

3

Yes I’ve tried converting the image tensor to numpy array by image.numpy() but it throws an AttributeError: 'tensor' object has no attribute 'numpy'

4

Googling that gives you this StackExchange article which includes addressing the error that you show.

5

Here’s a little experiment that I tried in the C2 W3 notebook:

a = tf.constant([[1, 2], [3, 4]])                 
b = tf.add(a, 1)
print(f"type(b) = {type(b)}")
print(f"b =\n{b}")

bNum = b.numpy()
print(f"type(bNum) = {type(bNum)}")
print(f"bNum =\n{bNum}")

When I run that, here’s what I see:


type(b) = <class 'tensorflow.python.framework.ops.EagerTensor'>
b =
[[2 3]
 [4 5]]
type(bNum) = <class 'numpy.ndarray'>
bNum =
[[2 3]
 [4 5]]

So it works for me. Based on the instructions in that StackExchange thread, my guess is that you probably don’t have EagerMode enabled.

The first step in debugging would be to print the type of your image object. What do you see?

6

Hello @paulinpaloalto
The image object is a tensor. When I applied image.numpy() on one image, it works perfectly fine. But when I run the dataset_augment function which I’ve quoted in the first comment, it doesn’t work and throws up the AttributeError.
EagerMode is enabled too.

7

Ok, so further debugging is required. The first step would be to show the actual exception trace that you are getting. Which actual operation is “throwing”? Then please actually print the type you are getting at the line that throws. As we can see from the type that I printed in my case, it’s a bit more complicated than tensor, right? There are lots of flavors of tensors. It might also help to see the current code you are running and exactly where you put the numpy() method. Note that in your template_matching function that you may need to do the reverse and convert the numpy array back to the correct type of tensor after the cropping, right? Because you’re processing the TF dataset with “map”, meaning that you’re assigning it back to something that will later be treated as a tensor.