Legal technology
Human in the loop: how to review AI's work
A practical review method for junior lawyers, with a step-by-step review process and four prompts you can copy.
The short version
To review AI's work as a lawyer, first decide how much checking the task needs by asking who will rely on it and what happens if it's wrong. Set the task up so the output is easy to check: ask AI for a source for every point, exact quotes only, a confidence score, and a list of what it didn't review. Then work through seven steps — write down what you expect, check it answers the question, start with the gaps and low-confidence points, check every point against the source, test every conclusion, compare it against your own list, and rewrite it in your own words. Finally, tell your supervisor that you used AI and what you checked.
When a partner reads our work, they assume we've prepared it, checked it, and can justify each part of it. Now that AI is becoming a regular part of legal work, that process is even more important.
Senior lawyers are usually better at this review process because they've already spent a few years delegating and reviewing the work of junior lawyers. But newer lawyers haven't had that experience and can end up in awkward situations where AI has taken the lead and the answer to why a certain position has been taken is "I don't know".
In this guide, I'll give you a practical method: how much checking each task needs, four prompts to make things easier to review, a seven-step review process, and a script for telling your supervisor what you checked.
1. Decide how much checking the task needs
Before I start, I decide how deep the review needs to go. The question I ask is: who's going to rely on this, and what happens if it's wrong?
| Task | Who relies on it | How to review it |
|---|---|---|
| Internal admin: a meeting agenda, file note or to-do list | You | Read it through once and fix anything that looks off. |
| A summary or internal work product | Your supervisor | Check every date, figure, quote and clause reference against the source. Spot-check the rest. |
| Anything going to a client, the other side or a court | Client, other side or court | Check every line against the source. Read every clause you rely on in full. |
| A task you haven't done before, or one where checking would take longer than doing it | Anyone | Do it yourself first (with guidance and reference materials from your team). Then, if you want, ask AI to critique your draft. |
Note: If you're not sure which row a task belongs in, take a conservative approach and go one row down.
The last row is the one that's easiest to skip. If you've never prepared a chronology yourself, you won't know what a good one looks like, and you won't be able to tell when AI's version is missing something. Doing it yourself the first time is how you build the judgement you need to review it later.
2. Set the task up so the output is easy to check
The easiest review is one where AI has already done some of the work for you. It has pointed you to the source, used the exact words, told you where it's least sure and told you what it skipped. There are four prompts that I have used quite a few times (or some combination of them) to make the initial drafts easier to check.
Prompt 1: Make it checkable
For every point in your answer:
- Give the clause, section or page number you relied on.
- List any assumptions you made, marked [ASSUMPTION].
- If the document doesn't answer something, write "Not addressed in the document". Don't fill the gap with general knowledge or what's usual in the market.
Set out your answer as a table with these columns: Issue | Summary | Source | Confidence.
Every point now has a source you can open. The "Not addressed" and "assumption" instructions help us avoid hallucinations because those are the areas where AI tends to fill in the blanks with the general information that it was trained on instead of the source material for our specific task.
Prompt 2: Use only the exact words in the reference material
When you quote the document, use only the exact words in the reference material, in quotation marks, with nothing added, removed or reworded. If you're summarising, don't use quotation marks. If you can't find the exact words, write "No exact wording found".
In my experience, AI likes to tidy up wording, and a tidied quote can change the meaning of a clause. This prompt makes it clear when you're reading the contract and when you're reading AI's paraphrasing of it.
Prompt 3: Give me a confidence score
For each point, assign a confidence score from 0 (not confident – needs careful reading) to 100 (very confident – reflected in the exact words of the source) that I can use to prioritise my review of your work. The score should reflect how confident you are that the point is accurate and fully supported by the document. In one line, say what would make you less confident.
Depending on the task, it can be helpful to use a confidence score as a starting point to see which areas to prioritise. This doesn't mean we can completely skip checking the parts with high confidence scores, but we can deal with the trickiest areas first.
Prompt 4: What did you not review?
List everything you did not review, or only partly reviewed. Include clauses, schedules, defined terms, documents referred to but not provided, and anything that was too long to read in full.
Then, for every clause you summarised, list any clause that qualifies it (for example, "subject to", "despite", "except" or "provided that") and confirm whether your summary takes it into account.
Long documents can get skimmed or cut off, and AI won't always tell you unless you ask. Its answer is a starting point for your review. It can get this wrong too.
3. Review the output, step by step
Once you have the output, this is the order I'd review it in. For a task in the second or third row of the table above, I'd work through every step. For internal admin, Steps 2 and 7 are usually enough.
Step 1: Write down what you expect before you read it
Re-read your supervisor's instructions and jot down the three to five points you'd expect the answer to cover. For a contract, skim the clause headings. It usually takes a few minutes, and in my experience it's the most reliable way to notice what AI has left out. You can use this as a baseline to assess the draft output.
Step 2: Check it answers the question you were asked
Compare the output with the instructions. Is it acting for the right party? Is it in the right jurisdiction? Is it the right scope, format and length? Has it taken into account the specific information that you've given it about the matter? AI can drift towards a more general question than the one you asked.
Step 3: Start with what AI didn't review and where it's least sure
If you used the confidence prompt, verify the lowest confidence scores first. Anything AI didn't review, either review yourself or flag to your supervisor.
Step 4: Check every point against the source
This is the core of the review:
- Open every reference. Does the clause, case or section exist, and does it say what AI says it does? For legislation and cases, check they're current and from the right jurisdiction.
- Search for every quote. Copy it and use Ctrl+F in the source document. If the search finds nothing, it isn't a quote.
- Read the whole clause. Then follow every "subject to", "despite", "except" and defined term to where it leads. A summary of clause 8.2 that misses "subject to clause 8.4" can miss a cap on liquidated damages entirely.
- Check the details. Names, dates and figures all need checking, and recalculate any maths yourself.
Step 5: Test every conclusion
For each conclusion, ask: "Which words in the source say this?" If you can't point to them, it's AI's opinion. Either label it as an open question to verify in more detail or take it out. Be especially careful with absolute words like "cannot", "must", "always" and "no right to". AI can turn a source's silence on a point into a firm answer.
Step 6: Compare it against your list from Step 1
Go back to the points you wrote down. Anything on your list that isn't in the output either needs adding or needs a reason for leaving it out. If something in the output surprised you, check it again.
Step 7: Make it yours
Rewrite it in your own words and your firm's style, and take out anything that sounds like AI. Then do a final test: could you explain every point to your supervisor, and why you took that position, without looking at your notes? If the honest answer to any part is "I don't know", that part isn't finished yet.
What each step catches
| Step | What it catches |
|---|---|
| 1 and 6: Your own list | Points AI left out |
| 2: The instructions | Answers to the wrong question |
| 3: Not reviewed and low scores | Gaps AI has told you about |
| 4: The source | Made-up references, paraphrased quotes, missed "subject to" clauses, wrong figures |
| 5: Conclusions | Opinions presented as answers |
| 7: Make it yours | Work you can't justify |
4. Tell your supervisor what you checked
From a supervisor's perspective, it is helpful to flag when AI has been used and your approach to checking it. Often, it's the draft that doesn't mention AI use but has some AI-sounding phrases that draws red flags and discomfort.
Say you've used AI to summarise the key risks in a supply agreement between your client, ABC Constructions, and XYZ Precast. Here's a cover email you could send:
Hi Jane,
Attached is the risk summary for the XYZ Precast supply agreement. I used [AI tool] for a first pass and then reviewed it against the agreement. I've checked every clause reference and quote, and read clauses 8 to 14 in full.
Two things to flag. The liquidated damages cap in clause 8.4 means ABC Constructions' protection runs out after about 19 days of delay. The agreement also doesn't deal with set-off, and I think that needs more thought.
I haven't reviewed Schedule 2 (Technical Specifications). Happy to talk it through.
[Name]
If it's a quick task, a line in Teams is enough:
Hi Jane, I used AI for the first pass on [task] and checked [what you checked]. The bit I'm least sure about is [X].
5. What to do now
Pick the next task you'd normally give to AI. Decide which row of the table it belongs in, consider using one or more of the prompts to help make reviewing easier, and work through the seven steps.
AI can write a first draft in seconds. Checking it properly is still our job, and it's one of the best ways to get better as a lawyer even when using AI.
Frequently asked questions
How much should a lawyer check AI's output?
It depends on who will rely on it and what happens if it's wrong. Internal admin only needs a read-through. For a summary or internal work product going to your supervisor, check every date, figure, quote and clause reference against the source and spot-check the rest. For anything going to a client, the other side or a court, check every line against the source and read every clause you rely on in full. If it's a task you haven't done before, do it yourself first and then ask AI to critique your draft.
What prompts make AI's work easier to check?
Four prompts help. Ask AI to give the clause, section or page number for every point, mark its assumptions and write "Not addressed in the document" instead of filling gaps. Ask it to quote only the exact words of the reference material. Ask for a confidence score from 0 to 100 for each point so you can prioritise your review. And ask it to list everything it did not review or only partly reviewed, including any qualifying clauses.
Should you tell your supervisor when you've used AI?
Yes. From a supervisor's perspective, it is helpful to flag when AI has been used and how you checked it — a draft that doesn't mention AI but has AI-sounding phrases is what raises red flags. A short cover email or a line in Teams is enough: say you used AI for a first pass, what you checked against the source, what you haven't reviewed, and the part you're least sure about.
Part of the Field Guide
This is 1 of 110+ guides
The full library adds templates and learning tracks — one payment, lifetime access.
Get the next guide in your inbox
One short, practical email most weeks. No spam — unsubscribe any time.