@astraleureka@social.treehouse.systems
Post #1753537
2026-04-19 06:03 UTC
@wren6991 this absurdity is on-par with the "claude re-emits vaguely json-shaped output repeatedly until the linter says it's valid" discovery from the recent source leak
Replies (1)
-
@wren6991@types.pl 2026-04-19 06:11
@astraleureka I don't know what Anthropic do but llama-cpp (open-source inference) apparently does masked decoding for tool calls. It recognises a magic token indicating the start of a tool call and from that point it forces probability to 0 for tokens that don't match an FSM for JSON syntax + tool call schema. This is done at inference level and might not be visible in the Claude leak, which afaik was just the harness. So it's not quite as dumb as I made it sound because the LLM is constrained to only produce syntactically and schematically correct JSON during tool calls. It's still funny that it just... types the JSON though