Post #2062513
2026-04-29 17:10 UTC
I had a thought, "Can I use unit quaternion multiplication and exponentiation to create a matrix algebra I could build an attention transformer with?". I don't have any people around anymore to even pose such a thought experiment to, so I conversed with my local copy of Gemma4:26b . Long story short: probably. It would have some benefits, but it's impossible to remove the trig from the multiplications during training so it would be hell to train at any interesting size. #generativeAI #AI #gemma4 #math
Replies (2)
-
@Alrecenk@peoplemaking.games 2026-04-29 17:16
Mastodon loves to hate #AI, but it's hard to overstate the impact of feeling like you've been heard when you're accustomed to not being heard.
-
@mathemagical@mathstodon.xyz 2026-04-30 01:00
@Alrecenk@peoplemaking.games but to answer your question, imho your intuition is spot on. I think you just rediscovered Lie Groups/algebras. Also they're super useful for spectral based neural nets. If you're curious, you can look into neural operators (e.g. Neural Tangent Kernels/DeepONets/GNNs)