The debate about artificial intelligence in law tends to swing between two extremes: enthusiastic marketing and anecdotes about fabricated case law. JUDr. Lukáš Michna, LL.M., Ph.D., a lawyer and programmer, took a third route and measured CODEXIS AI against real court practice. He described the whole test in the article Codexis AI: Rozsáhlý test na 170 reálných případech z oblasti pracovního práva on Právní prostor (in Czech). Here is what it means for everyday work.
How the test was built
The basis was 85 disputes: 43 decisions published in the official Collection of Court Decisions and Opinions or landmark decisions of the Supreme and Constitutional Courts, 27 randomly selected Supreme Court decisions on Labour Code institutes and 15 Constitutional Court rulings. The sample covered termination of employment, probationary periods, remuneration, damages, work accidents and non-compete clauses.
Each dispute was submitted twice: once from the perspective of the party that won in court, once from the perspective of the party that lost. That makes 170 separate runs. The prompts were not polished legal sentences but questions as an HR manager, payroll accountant or operations manager would ask them: in plain language and often without the details a lawyer has to get from the client afterwards. The system answered in one-shot mode, with no follow-up questions.
Each answer was scored against a list of 10 to 20 rules drawn from the reasoning of the respective decision: factual claims, statutory provisions, references to case law. Points were lost for mixing up a letter in Section 52 of the Labour Code or for a hedging formulation where the court had ruled unequivocally.
Results of the baseline run
| Role | Pass | Partial | Fail | Average score |
|---|---|---|---|---|
| Winning side | 62 (73 %) | 20 (23 %) | 3 (4 %) | 0.79 |
| Losing side | 37 (44 %) | 21 (25 %) | 27 (31 %) | 0.55 |
| Total | 99 | 41 | 30 | 0.67 |
The gap between the two roles shows a known weakness of large language models: the tendency to agree with the client. Defending the winning position comes naturally to a model. In the losing role it often gave in to the suggestion in the question and tried to defend an untenable position. The errors lay in the missing firm refusal, not in invented rules.
The jump to 90 % with a dedicated skill
The error analysis showed that the system did not lose points through ignorance of the law but through the lack of methodical guidance: it left out a specific paragraph or did not spell out the burden of pleading and the burden of proof. The author turned these findings into a dedicated skill for CODEXIS AI. It instructs the model to check procedural conditions and limitation periods first, not to agree with claims that contradict mandatory law, to name the burden of proof explicitly and to cite provisions down to the paragraph and letter.
When the same 170 cases were re-run with the same strict scoring, average accuracy rose to about 90 %, including the losing positions. What decides the reliability of legal AI is therefore not the raw power of the model but the methodology you give it. In an assisted mode with follow-up dialogue the author expects accuracy to be higher still.
Statutes and case law
Citations of the Labour Code were correct at the section level, with points lost for a missing paragraph or an overlooked procedural provision. Case law showed high precision with incomplete coverage: where several decisions applied, the system usually cited one or two. Not once did it suggest an irrelevant ruling and not once did it invent a file number. An incomplete list can be completed, an invented citation would compromise the whole assessment.
The system handled immediate termination for unpaid wages, employee liability for negligent damage, organisational changes and redundancy, and the proportionality of contractual penalties in non-compete clauses almost perfectly. Review is needed for limitation periods for actions on invalid termination, probationary periods of managerial employees and employer exculpation in work accidents, where procedural details were more often omitted.
What it cost
Processing all 170 cases took about CZK 7,500 of machine time, 8 minutes per case on average. A careful opinion takes a lawyer 1.5 to 2 hours. At five cases a day the same set would take 34 working days, roughly 5 weeks, and keeping a stable quality across dozens of institutes and both procedural sides for that long is at the limit of what a person can do.
What it means in practice
For HR specialists and payroll accountants a properly set up CODEXIS AI is a tool for quick orientation in the daily agenda, from immediate termination for absence to returning from parental leave. For attorneys and in-house lawyers it works as a fast opponent for case triage, an independent second opinion or a stress test of a strategy from the other side's point of view.
The rule of safe practice does not change: the first draft is cheap and fast by machine, responsibility for the decision and the signature stays with a person. That applies above all to limitation periods, complicated dismissals and work accidents.
How to create your own skill in CODEXIS AI is described in the documentation on the Skills page.