Models Are Getting Really Good At Git
I built GitBench because I was curious about how well models did git things. It turns out that it was a good idea because some models will surprise you. Some of the frontier models were remarkably bad and some cheap open models punched above their weight class. But, as they say, “the times they are a changin’”. Before I go too deep into the latest results, you may want to read up on GitBench and what it is and why I built it in the previous blog post.