Post #2437623

2026-04-27 11:32 UTC

@UlrikeHahn@fediscience.org @mxp@mastodon.acm.org Good point. I see this as one of the tensions in the technology. Current 'general purpose' models need very careful prompting to do anything worthwhile, but the vision is to have models which will produce human-like results from the sorts of vague, context-sensitive, and mutual knowldge presupposing instructions we give human assistants. So no one can decide which prompting styles are relevant for evaluation of capabilities.

Replies (1)

@UlrikeHahn@fediscience.org 2026-04-27 11:44

@tomstoneham@dair-community.social @mxp@mastodon.acm.org hmmm… feels more like a potential problem with the method of the paper to me? the conceptual core is the idea that all operations are reversible and symmetrical so do not give rise to order effects, but I don’t see how that could be the case? unless the reverse operation is strictly “undo the specific Rembrandt lighting you added” removing Rembrandt lighting need not take you back to the same place, and there can then also legitimately be order effects…. and *if* the reverse operation is “undo the specific Rembrandt lighting you added” those “undos” would necessarily *only* work in reverse sequential order (by definition) either way I don’t understand how the actual test lives up to the expressed constraints! or am I missing something?

Open ##2437624