Post #3324287

Last week I finally had an experience with AI agent coding, with Copilot 'cause it was free. What I wanted: Parse FoamFiles (which I described a week ago) using PyParsing. I know it can be parsed 'cause the format is not that much different from JSON with a few tricks, and PyParsing have a full JSON parser as an example. So I asked Copilot: "Write a PyParsing line to parse a 2; as {'a': '2'}". And it did. Which was nice. Then I started to add the rules: attributions with 2 values; explicit lists, dictionaries... And it keep increasing the number of lines, including some weird functions along the way. But it did parse. Then I decided I wouldn't waste time including every single rule and weird construction one by one: I changed the code to a full blown FoamFile and the expected Python result, with an assert of assert parsed(foam_file) == expected. And Copilot start chugging. And chugging. And it kept failing the assertion, and it keep adding code. And it kept failing the assertion over and over again. At the 5th time I got a failure, I had a hard look at the code: There were two weird functions, including one with something along the lines of if isinstance(input, list): if isinstance(input dict): ... ... which is completely bollocks and useless. In no time Copilot decided to use Groups, or even the specialized methods for lists and dictionaries -- which do the heavy lifting of building the specific Pythons structures directly. So yeah, it went from "Wow, that's really cool" to "Why it is producing so much garbage?" in less than 20 minutes.