Google's AI learns what "on-brand" means by quizzing you on hard cases

The short version

• Google Research and Google Ads built Agile Deliberation, a way to teach AI a fuzzy judgment, such as "healthy food" or "misleading ad", by showing a person borderline examples.
• In 18 live sessions, it produced more accurate AI than writing instructions by hand, which often made things worse.
• Every participant preferred it, and the code is free for anyone to use.

The problem it solves

Teaching AI to spot a stop sign is easy. Everyone agrees what a stop sign looks like. Teaching it to spot "healthy food" or "on-brand imagery" is much harder. Even the person asking can't list every rule upfront.

Today most teams handle this by writing instructions for the AI and tweaking them by hand. Google's researchers found this tiring. People had to guess which searches might turn up tricky cases. They also worried that fixing one mistake would quietly break something they had already fixed.

How Agile Deliberation works

The method runs in two stages.

First, the AI helps the person break a broad idea into parts. "Healthy food" might include salads, grain bowls and lean proteins. The person also names what to leave out, such as processed food.

Second, the AI hunts for borderline images that test those rules. It might show a fruit smoothie. Is that healthy because it has fruit, or out because of the sugar? The person decides. The AI then updates its own instructions to match.

The person never edits the instructions directly. They just react to examples, and the system does the rewriting.

What the tests showed

Google ran 18 live sessions of 1.5 hours each with real participants. They tried two subjects. One was healthy food. The other was a real content review job: spotting clickbait images in games and entertainment that promise misleading rewards.

Agile Deliberation was compared with three other approaches. These included writing one instruction and stopping there, and the slow manual method of editing instructions in spreadsheets.

It produced more accurate AI in both subjects. The biggest gains came on the hardest, most ambiguous images. The manual method often failed to improve accuracy, and sometimes made it worse.

People also felt better doing it. They reported less effort and less stress. Every participant said they preferred it to the manual methods. Experts said the borderline images showed them gaps in their own rules that they hadn't thought of.

Google has released the code free on GitHub, with a demo anyone can try.

What it means for brands in Indonesia

Marketing teams make fuzzy judgment calls all day. Is this creator's post on-brand, or does this comment cross the line? AI can now handle much of that review, but only if it learns your standards.

Agile Deliberation suggests a better way to teach it. Skip the long rulebook. Show the hard cases to the person who owns your brand standards, and let the AI learn from each decision.

The Google Ads link matters too. Ad platforms use similar systems to decide which ads pass review. When one of your ads is rejected, the reason is often a borderline call of exactly this kind.

What to do next

• Pick one review task your team does by hand, such as checking creator content against brand guidelines.
• Write down the main idea and its parts. Note what is clearly out, before involving any AI.
• Collect a few dozen borderline examples your team has argued about, and record the final decision on each.
• Use those examples to test any AI review tool, and judge it on the hard cases, not the easy ones.
• Let the person who owns the standard make the edge-case calls, and keep their decisions as the record.