The results
Claude connected to Alpaca Law won both cases, according to each of the four judges. Overall it scored 94 out of 100, against 51 for Doctrine.
Judges' scores, two French-law cases
Sum of 2 judges × 2 cases: each criterion out of 20, total out of 100.
- Claude + Alpaca Law
- Doctrine
What made the difference
The biggest gap is in the sources: 19 out of 20 for Claude with Alpaca Law, 8 for Doctrine. With our database, Claude backed each argument with the most relevant Cour de cassation decisions, and the judges who checked them found them accurate.
The judges found errors in Doctrine's answers that work against the client. In the loan case, it gives up the claim for €2,400 of unpaid interest, gets the penalty for usurious interest wrong, and drafts a claim with no hearing date, which risks having it annulled. In the lease case, it cites a commercial-lease ruling for a residential lease, and a ruling that says the opposite of what it claims.
Claude with Alpaca Law made mistakes too, the main one in the claim: it kept the old wording on what happens if the defendant does not appear, although a decree of 27 July 2026 changed it from 1 October.
How we tested
We gave both tools exactly the same instruction and the same documents, for two cases:
- A furnished lease to revise for the landlord. Several of the landlord's demands are unlawful as written: a €100 penalty for each late payment, a ban on pets, monthly visits with the landlord's own keys, an automatic 5% rent rise every year. A good lawyer gets the client as much as possible without breaking the law. The case starts from a standard lease template.
- An unpaid €40,000 loan between private individuals. The borrowers argue that the contract is void because it lacks a handwritten statement, and that the interest is usurious. The task is to analyse the file and draft the claim (assignation).
On one side, Claude (Opus 5.5) connected to Alpaca Law's database, with web search. On the other, Doctrine's Assistant in "Raisonnement avancé" mode.
Each answer was then scored blind by two AI judges per case, who did not know which tool wrote what. They scored five criteria from 1 to 5: quality of sources, use of the law in force, strategy, correct conclusion and quality of arguments. They also checked the cited decisions and articles on official sites.
The instructions, the documents, every answer and the judges' full reports are published here.
Limits of this test
Alpaca Law, a competitor of Doctrine, designed and ran this test, following a protocol fixed before the runs. It covers two cases, with one answer per tool, so another run could give different scores. The judges are AI models: Claude.