Post #2270884

@kirakira@furry.engineer @jneen@unstable.systems I used an LLM to write a test suite for some code I wrote myself. I also asked it to keep in mind the spec the code is supposed to implement. The LLM figured out how to achieve almost 100% MC/DC coverage using only the public API. It (correctly) justified condition that could not be tested. Some of the tests failed, and when they did, the LLM pointed out that the problem was with the code. In one case, it wrote a 2 line fix (add missing Vec::clears) I used directly. In one case, it suggested a 1 line change (use the correct Rust struct). In the others, I fixed the code myself. I didn’t do this because tests are unimportant. I did it because writing a test suite with good coverage by hand is very difficult. Is this a silver bullet? Of course not. I still need to cleanup and validate the test suite, and I need to write integration tests. But it saved me a lot of time, and resulted in much better test coverage. If nothing else, this makes regressions much less likely.