Has anyone thought about researching the behaviour and properties of a neural network where one or more outputs of one or more layers is routed back to the input of previous layers?
Could this be similar to how neurons in the human brain are connected?
Hi Stephen,
For example in Recurrent Neural Networks (RNNs) the hidden state is passed back into the network at the next time step. Some Convolutional Neural Network (CNNs) architectures use explicit feedback from later layers to earlier convolutional layers.
Skip (residual) connections which bypass one or more layers and directly connect an earlier layer to a later one are widely used in modern architectures such as GPT.
Shared weights, when the same set of parameters (weights) is used across multiple steps or layers in a model are also widely used.
Hi Pavel,
That’s interesting.
I’m asking this question because I’m fascinated by how neurons work in the human brain to create consciousness and self-awareness.
I’m wondering if consciousness and self-awareness in the human brains arises because of the interconnectedness of neurons and if perhaps neural networks could be interconnected in a similar fashion to reproduce consciousness and self-awareness in a machine.
hi @ai_is_cool
Your question is already asked many times by previous learners as it bound to raise this thought of mimicking human brain.
In human brain, consciousness and self-awareness is through right hemisphere, different parts of cerebral cortex as well as networking between brainstem, thalamus and posteromedial cortices. So you see how complex our human brain requires different part to works for a person to have consciousness and self awareness, it is not just merely interconnectivity of neurons.
AI lacks consciousness and self-awareness as the model created is trained on data fed based on some data distribution. Although to some level the transfer of information is tried to mimic but is based on statistical analysis and technical (maths included) to find why and how data gathered are interconnected.
Regards
DP
A single biological neuron is more complex than an artificial neuron, and simulating it can require about thousand of artificial neurons across 5-8 layers. Along with complex brain structure, as @Deepti_Prasad mentioned, as well as different learning process it is difficult to come up with a direct one-to-one mapping. I think to truly replicate the neural complexity of the brain, we would likely need not just more neurons or more layers and connections, but a different architecture.
Is the operation and function of a single biological neuron fully understood by science?
The safe answer is “probably not”, because “fully understood” is a very high bar.
functioning of neuron in brain is surely understood long back, from how different parts of neuron play significance in transmitting the information to use ion channels (sodium and potassium) present in neuronal membrane create electric signals to releas of neurotransmitter chemical messangers from axon terminals (end points of axon) to the synapse (Gap between the neurons where communication occurs)
With that being said, human brain itself is a complex structure, and mimicking just a neuron doesn’t mean AI can mimick human intelligence, as Tom mentioned it is still a far cry and as a person who have little understanding of both world, I honestly wouldnt want AI to work as human intelligence, that day would be disastrous for humanity which few technology people still don’t understand.