Which models are good at git? Understanding GitBench
In this video I go over what GitBench is, how to read its results, and some of the interesting things we can see in the results themselves.
Feel free to check out GitBench at https://gitbench.gitkraken.com
GitHub repo: https://github.com/gitkraken/gitbench
Chapters:
- What is GitBench? 0:00
- Reading the Charts 3:02
- Anthropic Models 6:16
- OpenAI Models 8:51
- Reasoning Levels 11:46
- Token Usage 13:58
- Wrapping Up 16:58
For some reading on GitBench check out these two blog posts:
- Introducing GitBench: https://gitkraken.com/blog/introducing-gitbench
- Models Are Getting Really Good At Git: https://gitkraken.com/blog/models-are-getting-really-good-at-git
GitKraken Desktop:
http://tr.ee/GKDYT
Kepler (ADE):
https://www.gitkraken.com/kepler
GitLens for VS Code:
http://tr.ee/GLYT
Git Integration for Jira:
http://tr.ee/GijYT
Git Blog:
http://gitkraken.com/blog