Post #3844858

2026-07-15 23:21 UTC

I see that a SaaS product I use regularly has started using genAI for global search. I can tell because when I type in the exact name of a document I know exists, it no longer shows that document anywhere on its list of responses. When I type “incident response manual”, I'm not looking for the Customer Service Manual or the Risk Management Manual. I wrote all these manuals, so I'm pretty certain of this.

Replies (1)

@isaacfreeman@cloudisland.nz 2026-07-16 00:31

I don't really blame the LLM here. The real cause is the underlying attitude towards users. It would be easy enough to check for a match to the exact text first, then add additional results for fuzzy matches from an LLM. That might even sometimes be useful. But it would require thinking about was users might actually want to do. In my case, the global search field in this product wasn't primarily used for fuzzy search. 90% of the time I used it to bring up the exact document I wanted to see. The other 10% of the time I wanted to do a global search-and-replace to update specific text consistently across documents. Now neither of those tasks are possible, but I can type stuff and have an LLM surface (supposedly) deeper connections between documents. Even if that ever works, it's a different task and there was already a chatbot for it that I have never used, and they've broken the tasks I do most frequently.

Open ##3846075