Elektrine lite

← Feed

@neuralreckoning@neuromatch.social

Post #3195789

2025-01-23 17:11 UTC

@GabrielBena@neuromatch.social By varying the number of connections between the modules we can fully span the possible range of structural modularity (measured with the widely used graph-theoretic Q metric), train on the task using backprop, then measure how much each module specialises on its own inputs. Good news! Firstly, all the measures of specialisation qualitatively agree, so we're measuring something real 🤞. When the two modules are fully connected to each other, we don't see any specialisation and when they're maximally modular we do. This is what we'd expect. 😅 All good then? Well... What surprised us is how much structural modularity you need before you observe specialisation. You need Q>0.4, higher than you observe in the brain. So does this mean that structural and functional modularity are unrelated in practice? Not necessarily, there could be other mechanisms at play.

Replies (1)

  • @GabrielBena@neuromatch.social Our intuition suggested that resource constraints are likely to be important: there's little incentive to specialise if you have infinite resources. Sure enough, when we did large parameter sweeps we see that you get more specialisation when resources (neurons, synapses) are tight. This seems like an important insight: we see that resource constrained biological brains are great at generalisation, an expected outcome of having specialised modules with generalisable functions, while machine learning systems are not. Maybe we give them too much computational power? 🤯

    Open ##3195790