How to input coordinates to neural network?

[mosofsky]

I’m wondering how to input geographic coordinates into a neural network, for example (x,y) coordinates or latitude/longitude, etc. I guess a neural network could learn that the components of a coordinate are related. But since I already know that, wouldn’t it help if I could tell the neural network? I can’t figure out how this might be done.

In particular, suppose I have training data with multiple coordinates and the prediction is also a coordinate. The domain I’m thinking of is indoor positioning where a phone reads signal strength from beacons with known locations and from that tries to infer the location of the phone. Let’s say the input data is (x1, y1, x2, y2, x3, y3, … xn, yn). x1 and y1 are the known coordinates of one beacon. Wouldn’t it be good to tell the neural network that x1 is related to y1 so it doesn’t bother considering the possibility that x1 is related to y2, for example?

I wonder if Convolutional Layers could be used, which are discussed in the lecture Additional Layer Types? It seems like I could define windows for the first 2 features (i.e. x1 and y1), the second 2, etc. But how does a Convolutional Layer combine the features? Does it average them? Sum them? Multiply them? Concatenate them? Is the method of fusing features configurable?

I tried to find the answer on my own and came across an article that mentions Feature Crosses. If I understood correctly, Feature Crosses allow you to tell a neural network that a latitude feature and a longitude feature belong together.

However, the article doesn’t provide a link explaining how to implement Feature Crosses with a Keras layer, it only provides a link to a TensorFlow glossary definition.

I tried searching for “keras feature cross” but could only find an unanswered question on stackoverflow: How use to Feature crosses with Keras Functional API.

So, my questions are as follows:

1] Generally, how do we inform a Neural Network about known relationships among input features? This is a general question where geographic data is just one example. I’m also curious about things like image classification where it seems important to know which pixels are close. Seems like Convolutional Layers are the answer, but could you please confirm?

2] Specifically, how do I input geographic coordinates to a Neural Network so that it’s clear which x coordinates belong to which y coordinates?

3] Even more specifically, where is a link to Keras documentation for the previous question?

Thank you in advance!

[TMosh]

Lat/Long coordinates are just two floating point values. They’re very easy to use.

You’re over-thinking this.

The NN will learn anything it needs to to minimize the cost. You don’t have to try to impart your human-based knowledge.

[rmwkwok]

Hello Michael,

(1) I can’t confirm convolutional layer will work nor it is a good solution for the objective in question. The only thing for sure is if your input data is (x1, y1, x2, y2, x3, y3, … xn, yn), you may use a convolutional layer with a window of 2 and a strike of 2 so that it focuses on a pair of coordinates at a time. It will be doing activation(w1x1 + w2y1), activation(w1x2 + w2y2), and so on. Although this works on a pair of coordinate at a time, it doesn’t necessarily make sense.

(3) tf.feature_column.crossed_column  |  TensorFlow v2.9.1

Raymond

[rmwkwok]

Michael, I forgot to talk about this. Would you consider to use the signal strength data as the input for your convolutional network instead? So you don’t use the coordinate values as input, but divide the room into a rectangular array of grids where each grid has a strength value. I think this looks more promising.

[mosofsky]

Thanks for the idea @rmwkwok . Let me see if I understand. Let’s say the room can be divided into a 4 x 4 grid. So when I call fit, for x, I pass 16 strength variables, one for each cell in the grid, and for y, I pass a value that ranges 0-15 corresponding to the 16 cells in the grid.

Ultimately, the application needs to render a blue dot at a geographic coordinate. So I guess I could map the 16 cells to a lat/long at the center point. If the model predicts cell 5, I could render the blue dot at the center point of cell 5. To get more fidelity I could increase the number of cells.

[rmwkwok]

Hi Michael @mosofsky,

The idea about the input is as you said. It can be a 4 by 4 array if you would like to use convolutional layer, or a 1 by 16 if you would like to use the dense layers that we have learnt in MLS.

As for the output, it can be a dense layer with 16 neurons plus softmax to predict the probability of being a blue dot. Ofcourse the 16 neurons each represents a grid.

In this arrangement, the lat-lon values never shows up but implicitly represented.

Raymond