AI lifted junior lawyers' drafts in a 3-month trial. Their judgment without AI didn't grow
A randomized trial with 133 patent lawyers found AI improved the work everyone delivered. Only senior lawyers came out better at the job when the AI was taken away.
Most studies of AI at work measure output while people use it. A new working paper from the US National Bureau of Economic Research (NBER), led by MIT economist David Autor with researchers at Google, measured something harder: whether 3 months of working with AI left people more skilled, or just more dependent.
The answer depended on seniority.
How the trial worked
The pre-registered study, revised in October, enrolled 133 patent lawyers at 11 US intellectual property firms. About two-thirds got a custom AI drafting assistant built by Google Labs. The rest worked with their usual tools and no generative AI. Lawyers were assigned at random within each firm.
At 10 days and again at 90 days, everyone drafted a patent application, the AI group with the assistant. At 90 days, everyone also redlined a flawed patent draft without any AI, marking up what was wrong with it. That last task tests professional judgment. Blinded patent attorneys at an independent firm scored all of the work on 5 measures. Enforceability and technical accuracy were among them. Juniors were lawyers with under 7 years of experience.
Better drafts for everyone, better judgment only for seniors
With AI, drafts got better. The AI group beat the control group by 0.34 standard deviations at 10 days and 0.38 at 90 days, with larger gains for juniors. The gain came mostly from fewer weak drafts. AI also saved about 10 minutes on a drafting task that took the control group 110 minutes on average.
The unassisted redlining told a different story. Lawyers who had spent 3 months drafting with AI did better overall, by 0.32 standard deviations. All of that advantage came from senior lawyers, whose gain was 0.45. Junior lawyers showed no average gain at all.
Their scores spread out instead. Juniors who had used AI produced fewer middling markups, more good ones and more poor ones. The authors read this as AI acting as a springboard for some juniors and a cushion for others, and say plainly that the study cannot tell why.
With AI, juniors drafted better than seniors without it: 0.36 against 0.13 at 90 days. The juniors also believed the tool was making them better. It was, as long as the tool was in use.
What it means for companies in Indonesia
The trial was about patent law, but the shape of the work is common: junior staff learn by doing a task under a senior's eye until they can judge it alone. Banks training credit analysts, agencies training strategists and IT teams training engineers all work this way. The authors argue that foundational expertise may be what lets a person turn AI-assisted work into lasting skill.
Measure skill without the tool. If you only look at AI-assisted output, a junior who is coasting and a junior who is learning look the same. The study shows those 2 groups exist side by side. Regular reviews done without AI are how you tell them apart.
Keep the apprenticeship. Give juniors tasks where they must find and fix problems themselves, and have seniors review the reasoning as well as the result.
Expect seniors to gain the most. In this trial, experienced lawyers gained the most lasting skill from the same tool. Plan the junior rollout with that in mind, because the same tool did not build the same skill in juniors.
What the study cannot yet say
The sample is small, the firms all do regular patent work for Google, which also built the tool and paid for the experiment, and 3 months is short next to the years it takes to build expertise. Firms were reluctant to join if it meant a baseline skills test. Without one, the result rests on random assignment. The authors call for longer trials that test people without AI at the start and again later. Until those exist, the safest reading is the one the authors give: judge AI by what it does to the people using it as well as by the work it turns out.