C3w2 lect video13 'Pruning'. Neuronal activity always sparse in brain ..?

At 1min 50 sec:

“Everywhere you look in the brain, the activity of neurons is always sparse.”

It would be nice to have a reference/citation for this statement.

Hi @shahin

well, In some way I would agree with you… going too far with the analogies on how our brain works are not really useful. But, you know, we as humans need to find an explanation that is close to our life, we need to make it “human”.

Happy learning.

Hi @luigisaetta,
I’m not sure it was stated by the lecturer as an analogy.
The activity of individual neurons in whole tissue/animal models has only relatively recently become observable. I guess structural sparsity was already documented, so I supposed functionality sparsity could be inferred from that to some extent. But I wondered if he was eluding to some specific observations/publication(s) when he stated this and whether it was specifically the inspiration for pruning in artificial neural networks, because on skimming through the Le Cun paper, I didn’t see a reference to any. Maybe it is in Le Cun’s PhD thesis … ?

(I’m wondering if it really was inspired by biological observations, in the same way that ConvNets were apparently inspired by receptive fields in retina and ResNets by pyramidal cells in cortex).

@shahin

personally, I don’t think the analogy between Neural Networks and the way our brain works should be pushed to far.

bye