Same AI, 40 Legal Tasks, Yet 3 Scores
Last month, a research lab called LN Labs published a legal AI evaluation study. They took one AI model, Claude’s Opus 4.8, and ran it through forty legal tasks in the data protection and operational resilience domain, such as checking a data agreement against a set of rules. Then they ran the same model three times, changing only the system around it: plain Claude Chat, then Claude’s Cowork with a legal plugin, and finally an open-source setup, the famous MikeOSS.
By “the system around it” they mean everything that isn’t the model - the instructions it’s given, the documents it can reach, the steps it works through, the criteria of how the answer should come out.
Same model all three times, and the scores? The Claude Cowork + legal plugin setup passed 83% of the tasks. Plain Claude Chat passed 76%. MikeOSS passed 73%, below plain chat. That same Cowork + legal plugin setup also cost less per task than plain Claude Chat, 80 cents against $2.80, because it reached a good answer in fewer tries.
So the model is only part of the story. The bigger difference came from the system around it - the instructions, the documents it could pull from, the steps it followed.
Rent the model, but own what’s around it.
Microsoft Made the Model a Slot
On June 16th, Microsoft made Copilot Cowork generally available worldwide. Copilot Cowork is the agentic part of Copilot that takes on longer, multi-step work across your apps like Word, Outlook and Teams, and runs it end to end. It’s Microsoft’s answer to Anthropic’s Claude Cowork.
One difference is that Microsoft built it from the start to run on more than one model. Currently it runs on Anthropic’s Claude models, while OpenAI’s GPT5.5 already available in the preview version, and Microsoft’s own model, Cowork 1, is coming soon.
Now, for anyone still wondering whether it’s worth the time to build your own skill files and playbooks for legal tasks, the way Microsoft Copilot Cowork works would suggest that’s a yes.
The same architecture we’ve seen with Claude Cowork and other agentic systems: skills and plugins sit on top of the AI model, separate from it. A skill is a set of instructions that makes the system work your firm’s way. A plugin bundles those instructions together with connections to your other systems. You build them once, and they run on whatever model is behind them.
Those plugins also move between products. Microsoft lets you take a plugin you built in Anthropic’s Claude Cowork and bring it into Copilot Cowork, in a few steps. So the work you put into that layer isn’t tied to the model, and it isn’t tied to the product you first create it in. A plugin created for one rival’s tool runs in the other’s.
So one of the largest software companies in the world built its product on the same finding as the LN Labs test. The model is the part you swap out. What you create around it is the part that makes the difference.
What about the cost? Well, by Microsoft’s own testing, with the same model (Opus 4.8), Copilot Cowork came out 30 to 40% cheaper per task than Claude Cowork run through its Microsoft 365 connector. It’s Microsoft’s own benchmark, so weigh it accordingly. But it’s the same finding from the top of this issue, now between two real products: same model, different system around it, different cost.
The Scarce Part
In June, we saw two more announcements, both from the tools lawyers already use.
The first is Microsoft’s Legal agent for Word. Microsoft is expanding it to Frontier users worldwide in Public Preview starting mid-June 2026, with general availability in early July 2026. You give it a contract, and it reads through it, points out the risky or unusual clauses, checks them against your firm’s playbook, and makes its edits as tracked changes, the way a junior lawyer would.
The second is Harvey, and not in a way you would expect. On 16 June Harvey announced it moved INSIDE Microsoft Copilot and Copilot Cowork. As you work, it pulls your firm’s past deals and positions from the Harvey Vault - the place where your team stores that material.
But each tool needs something from your firm. The Word agent checks a contract AGAINST your playbook, so it’s only as good as the playbook you give it. Harvey pulls FROM your Vault, so it’s only as good as the information sitting there. The tool itself is the “easy” part. It’s the same for every firm that uses it. What makes your firm’s answer better than the next firm’s is the know-how behind it - your playbook, your standard positions, the way your firm really handles a matter.
That know-how is the scarce part here. And it matters where it sits. Your playbook is yours, but Harvey’s Vault sits inside Harvey, held by the vendor.
The other problem is that in a lot of firms, your know-how mostly lives in the heads of a few senior people, not playbooks. And it walks out the door every evening at six.
Even the Model Maker Moved Up
On 23 June, Anthropic launched Claude Tag. Rather than opening Claude in the Desktop App or browser, you add it to your team’s Slack as a member. Tag it on a task in a channel and it goes off, does the work, and posts the result back. Everyone in the channel can watch and steer it, not just the person who asked. It can also run on its own, on a schedule or by watching a channel and flagging what matters. Anthropic says 65% of their own product team’s work is now created with their internal version of Claude Tag.
Now, the most interesting part of this new product is the memory. The longer Claude Tag sits with your team, the more it picks up how you work (the calls you’ve made, the way you run things) and it uses that to do better next time. Anthropic calls it the “tacit knowledge” that lets Claude give you its best.
So the company with one of the world’s best AI models built a product that sells something else entirely: a memory of how your firm works. The caveat is that the memory sits on Anthropic’s servers, inside their product, not yours.
Anthropic is unusually transparent about this. Their own guide says to keep the memory short, and to put your actual playbooks “in a repository Claude can read”, which simply means: files you own and control.
For now, Claude Tag works only in Slack and it’s still in beta. Anthropic says more apps are coming (hopefully Teams is next). So don’t switch anything on just yet, just notice the pattern: from Harvey’s Vault to Claude’s memory, every new tool wants to become the place your firm’s knowledge collects. And that knowledge is the most valuable thing the firm has - the only question is whether it piles up somewhere you own, or somewhere you rent.
Own the Knowledge, Rent the Model
That brings us to the quietest announcement of the month.
On 12 June, Google published the Open Knowledge Format, or OKF. It’s a standard way to write down what your firm knows - your playbooks, your positions, how you run a matter - as short notes, one topic each, in a shape every AI can read. People have kept notes for years; what’s new is the standard. Because everyone writes them the same way, any AI tool can pick them up and use them, and the notes stay plain files in a folder you own. It isn’t a product, there is nothing to buy, and no one to lock you in.
OKF didn’t come from nowhere. It’s the standard version of the LLM-Wiki idea from Andrej Karpathy, a founding member of OpenAI and one of the most respected people in AI today.
Today, when you ask AI something, it reads through your documents, works out the answer, hands it over and keeps none of the knowledge. Ask it another question and it starts from scratch, reading everything over. It never builds up any memory of your firm.
Karpathy’s idea is to let the AI keep notes instead: it writes down what it learns and keeps them up to date, so the knowledge builds up over time. OKF takes that idea and turns it into one shared standard, so different people’s notes and different AI tools can work together.
This is the layer you own. The AI model underneath can swap out whenever a better one arrives, and your know-how stays put, in a folder you own, not in Harvey’s Vault or Claude’s memory. Own the knowledge, rent the model.
To reinforce this point, on June 27th, Microsoft’s CEO Satya Nadella said there should be as many AI models in the world as there are firms, because a firm is a learning system and you can’t outsource your learning. Now let’s be realistic, building your own AI model is out of reach for a boutique, but the ground he’s standing on, your knowledge, organized and owned by you, is what OKF gives you.
OKF is the notes that make it possible for AI agents sitting on top of it to read the information that comes in as legal work is done, write it up, keep everything current, and flag what’s gone out of date, with a person signing off on each change.
Here’s the core idea:
And those agents can run in whatever tool your firm uses - Claude Cowork, Microsoft’s Copilot Cowork, OpenAI’s Codex, Gemini’s Gems. The knowledge is yours; the tool is just what operates on top of it, and you can change it any time.
To try OKF today, copy the Google’s OKF spec file, copy paste it to your AI (Claude, Gemini, whichever you use), ask it to summarize the information and ask it to tell you how you can use OKF in your firm.
Legal AI in Action
📰 My First Article for Global Legal AI
I’m honoured I’ve joined Global Legal AI, created by Jean Gan, as a contributing writer. And my first article is out! It takes on a question a lot of boutiques are quietly working through and one we dived deeper into in one of the previous newsletters: can Claude’s cheaper Team plan be trusted with confidential client work, or must the protections you need sit higher up on Enterprise? The article tests it against the three client duties that matter for a lawyer.
🎙 LegalTechTalk Interview 2: The Asset Under Every Tool
David Malkinson spent twenty years putting document systems into hundreds of law firms, first at Phoenix, then Morae, and now runs SIGNL, an AI advisory. We sat down at LegalTechTalk to cut through the buyer’s overwhelm: strip away every platform on the show floor and what’s left is the instructions, the skills files, the plain-English workflows underneath - and that, he says, is the whole game. We got into why those are the assets you carry between tools so you’re never locked in and why “data is the moat” has hardened into advice nobody can actually use.
A new interview drops every few days - watch on YouTube or follow the full series on my site.
🎬 Create Your Firm’s Second Brain
What Google’s new Open Knowledge Format actually is and why it’s so powerful for law firms, the one question you have to answer before you touch it, the Andrej Karpathy LLM-Wiki idea that lets an AI keep those notes current instead of letting them rot, and how the whole thing comes together so your firm’s know-how stops walking out the door every evening at six.
🎙 The Junior Lawyer Building Judgment While AI Takes the Grunt Work
George Hannah has joined a London firm Lewis Silkin at 18, straight out of school, on a solicitor apprenticeship and on the side he’s become one of the sharpest reporters of legal AI, writing the Best Practice newsletter alongside a podcast and a busy LinkedIn. He’s a junior sitting right inside the junior lawyer paradox. We talked through how juniors build judgment now that the grunt work is the first thing automated, his idea that legal AI should know your seniority and coach a junior instead of handing over a finished answer, and why, even with the model providers now coming for legal, a law firm still isn’t a tech company.
Coming up!
🎙 Next Tuesday at 2pm CET!
Next week’s guests on Rok’s Legal AI Conversations is Denisa Kopandi, a senior associate at top Romanian law firm who has run AI on every task - email, contract review, due diligence, every day for two years, in a civil law jurisdiction where almost nothing comes standardized.
We discuss why AI was slower than doing the work by hand for a long time, and why the “review 100 contracts instantly” demo falls apart contracts that run anywhere from 10 to 110 pages; how she prompts like “the mom who waits in the dark and turns on the lamp”, interrogating the model and back checking every answer in a clean chat against the law; and why she wants vendors that think less for her, not more - tools that assume the answer might be wrong and make it easy to verify.
Each edition of Legal AI Brief brings practical lessons from firms using AI safely.