Post #2342909
2026-04-30 01:00 UTC
@Alrecenk@peoplemaking.games but to answer your question, imho your intuition is spot on. I think you just rediscovered Lie Groups/algebras.
Also they're super useful for spectral based neural nets. If you're curious, you can look into neural operators (e.g. Neural Tangent Kernels/DeepONets/GNNs)
Replies (1)
-
@Alrecenk@peoplemaking.games 2026-04-30 02:01
@mathemagical@mathstodon.xyz Yeah, I started with a lie group, but if I use exp(log(A)+log(B)) as a commutative stand-in for the multiplication operation then I think it's a proper algebra. Still a lot of trig functions, but I can make the matrices behave properly. Also, If I only use real numbers in the key and query matrices and then replace the dot product with a more generic angle difference function that generates reals then I only have to raise quaternions to real powers. No quat^quat is ever required. Also rotatary position encoding is trivial because it's all already rotations 🤣 .