If one of the input features is continuous & others are categorical like in the demand prediction example with awareness. Can the input layer have multiple activation functions different for each of the neurons to process the data? I don’t believe that’s possible as one cannot specify the position of this particular feature in the vector explicitly nor there is a chance that we can have a Tf.Dense like layer where we can specify an activation function & config array rather than just the number of units. If this is answered elsewhere can someone please link that. Thank you
1 Like
This is what you need to read:
Also you can check the TFAT specialization, for the functional type model.
All of the inputs must be numeric values.
Inputs don’t have activation functions - the activations are only applied after the inputs are multiplied by a weight matrix.
1 Like
Your post seems to have two sides - on the one hand you were suggesting that a categorical feature and a numerical feature requires different activation functions at the input layer, on the other hand you were interested in using multiple activations in a layer.
First, as Tom pointed out, we do not apply activations to the input layer, but the hidden layers and sometimes output layer.
Second, if you want to have multiple activations in one hidden layer, although Tensorflow’s Dense layer will not allow that, you can concatenate multiple Dense layers up - each dense layer uses one activation. However, I do not think it is a usual approach.
Lastly, why would you like to use the activations for treating the categorical and numerical features in a different manner? We usually converts the categorical feature into numbers and not considering about activations.
Raymond
1 Like
Your last point answers my question, I was a bit confused there. But can you please elaborate on the approach in the second point, what would the topology look like. And is the article linked by Gent useful in realizing a model with this topology?
Eakanath
That article provided a good example.