← All editions
Edition 4 Thursday, 2 October 2025

Stop the Legal AI Tool Roulette

The headlines look good. But inside firms, the story is different

The Legal AI Brief MPL Legal Tech Advisors
Edition 4 · Thursday, 2 October 2025

This Week’s Theme

The headlines look good. Big funding, big partnerships. But inside firms, the story is different: legal AI tools don’t match workflows, don’t earn trust, and don’t save time.

The real problem isn’t the AI itself. Buying platforms before fixing data and processes guarantees failure.

This edition is the antidote. Clean your data, define the workflow, and run short, defensible pilots that prove value on your matters before rollout.

Stop the AI Tool Roulette

When tools underdeliver, it’s rarely because “AI doesn’t work.” It’s because the firm bought software before doing the groundwork.

Here’s the sequence that consistently works:

1. Fix the data first

One repository. Clear names, versions, folders. If the structure is messy, every output will be messy.

2. Map one workflow

Choose a common matter. Write the 6–10 steps. Mark what’s judgment and what can repeat. That’s your script.

3. Run a 30-day pilot

Use live but low-risk work. Log inputs and outputs. Senior review weekly. Decide: scale or stop.

4. Test vendors on your script

Same documents, same steps, same checks. Score on accuracy, auditability, speed, security, and repeat use.

5. Slot AI into tasks, not roles

Extraction, classification, checklists. Keep it narrow, supervised. No “act like a lawyer” prompts.

This turns hype into numbers. You’ll either reclaim hours and reduce errors, or you won’t. Either way, you stop paying for hope.

🎬 Upskill: Future Proof Yourself as a Lawyer

Build the skills to judge tools, spot weak workflows, and lead adoption decisions, so you’re not reacting to hype.

The 5-Step AI Checklist Every Law Firm Needs in 2026

🎬 Client Conversation: Building Confidence Step by Step

Hoorntje Legal’s managing partner on why we mapped workflows, triaged risk, and built confidence with small, controlled steps, before buying new platforms.

How This Legal Firm Got Clarity on AI Fast (Real Life Case Study)

Red Flag of the Week

Firms keep buying AI licenses before fixing their foundations. The result? Shelfware.

Expensive seats go unused because workflows weren’t mapped first.

Budgets get burned, and leadership assumes “AI doesn’t work” when the real issue is misfit, not the technology.

1. Usage beats logos

ARR and partnerships don’t prove reliance. Track repeat usage and matter coverage, or it doesn’t count.

2. Small wins > big platforms

Narrow, measurable tasks on your own documents beat “do everything” claims on accuracy, speed, and auditability.

3. Evaluation is a lawyer skill

Teams that scope, standardize, and score pilots make faster, safer buy/kill calls, and stop paying for “shelfware”.

Looking Ahead

🎙 This Saturday at 2pm CET!

This week’s guest on Rok’s Legal AI Conversations is Jason Marett, former Am Law Attorney, Founder and Researcher of The Recurial Codex Project.

Podcast guest cover
Clash of AI and billable model

Each edition of Legal AI Brief brings practical lessons from firms using AI safely.

← Previous Edition 3: The Craftsman's Dilemma: Why Lawyers Resist AI Next → Edition 5: Legal AI Policies Are Theatre
More editions

Get the next edition in your inbox.

Every Thursday. No noise, no pitch — just what's worth knowing about AI risk in legal practice this week.