About·benchmarks.law
Open benchmarks for legal documents. Come build them.
benchmarks.law measures how faithfully DOCX tools reproduce Microsoft Word, and how legal AI tools hold up against in-house lawyers. Every input, output, score and log is published, so any number here can be checked, reproduced or challenged.
/ Who keeps it
I'm Arthur Souza Rodrigues, a lawyer working between New York and São Paulo on contracts, legal operations and legal AI. I started the bench because lawyers live in Word, and the tools that promise to handle Word documents are rarely measured against Word itself.
Full disclosure: I also write jubarte, one of the engines compared here. That is why the scorer is lifted verbatim from an independent benchmark, every vendor runs on the same fixed document sets, and the leaderboard marks no tool as ours.
/ Contribute
Add a tool
If it converts DOCX to PDF or compares two documents without a GUI, it can be benchmarked. Open an issue with how to run it.
Open an issue →Send a document that breaks things
The fixtures are real documents. One that trips Word, an engine or the scorer is worth more than a hundred that don't.
Report it →Challenge a score
Think a number is wrong, or the method is unfair to a tool? Link the case and say why. Corrections land in the changelog.
Read the method →/ Get in touch
Using these numbers, building a tool, or just curious? Say hello.
Add me on LinkedIn, follow the work on GitHub and Hugging Face, or write directly. For consulting and legal work, the contact page has the details and a booking link.
- contact@arthur.law
- Based
- New York · São Paulo
- GitHub
- github.com/arthrod
- Hugging Face
- huggingface.co/arthrod