Post #1442135

2025-01-14 13:04 UTC

@marcusghosh In previous work, we found that when multimodal information arrives sparsely in time (e.g. prey hiding from predator), nonlinear fusion of different modalities gives a big improvement over linear fusion. https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012246 In this paper, we looked at what happens when, in addition to being sparse, information arrives in contiguous bursts (e.g. prey scurrying from hiding spot to hiding spot). In general, the optimal algorithm is computationally intractable, so how far can you get with simple neural architectures?

Replies (1)

@neuralreckoning@neuromatch.social 2025-01-14 13:06

@marcusghosh We compared the performance of linear and nonlinear algorithms that ignore temporal structure to two architectures that can use it. The first just uses a sliding window or fixed length short term memory. The second is a recurrent neural network, which in principle can have a much longer memory.

Open ##1442136