@neuralreckoning@neuromatch.social
Post #1442137
2025-01-14 13:07 UTC
@marcusghosh We were expecting the RNN to hugely outperform the sliding window approach, as it has a potentially longer memory and orders of magnitude more trainable parameters. However, if the bursts of information are not too long, the much simpler network does better.
They also differ in how they generalise. If you train on one burst length and test on other burst lengths, the sliding window algorithms generalise well to longer bursts than they were trained on, and poorly to shorter bursts. The RNNs simply generalise worse the bigger the difference.
Replies (1)
-
@neuralreckoning@neuromatch.social 2025-01-14 13:07
@marcusghosh When we tested more realistic mixed distributions of burst lengths using either a uniform or naturalistic Lévy flight distribution, the simpler algorithms tended to perform better.