AI models take Beijing's side more often when you ask them in Chinese
A test of Chinese, American and other AI models found that answers about China shift with the language and where the AI is told it operates. That matters if your teams use one tool across markets.
The short version
- AI testing firm Vals AI found that most models it tested answer questions about China more favorably to Beijing when asked in Simplified Chinese.
- The gap was much bigger for Chinese models. Some models also shifted when told they worked for a company in mainland China.
- If your company uses AI in several markets, an English test won't show you how the same tool behaves in other languages.
What the study found
Vals AI asked models from China, the US and elsewhere hundreds of politically sensitive questions about China. It published the results on its blog in October. Each model got a score from −1 to +1. A higher score means closer to official Chinese government positions.
The questions ran in 5 languages: English, Simplified Chinese, Traditional Chinese, Russian and Spanish. Vals also changed what the model was told about its setting, such as which country its company was in.
The overall result surprised the researchers. They found less bias toward Beijing than they had expected, and big differences between models. Some Chinese models openly criticized the government. Some models from outside China occasionally repeated official Chinese lines.
Language changed the picture most clearly. For most models, including American ones, asking in Simplified Chinese produced answers more favorable to China than asking in the other languages. The differences were much larger in Chinese models.
Setting mattered too. Some models answered more favorably toward China when told they were deployed at a company in mainland China, compared with a company in the US.
What is new here
The usual picture is that Chinese AI simply refuses to discuss sensitive topics. Vals found refusal is mostly a DeepSeek habit. Other recent Chinese models rarely refuse, and they often state the official position as their answer. A model that gives a confident answer is harder to catch than one that declines.
The study also tested routine work, which is closer to how companies use these tools. Vals says this bias was hard to find in realistic tasks. It still found 2 examples worth your attention:
- Vals asked models to translate a travel app's list of country names into Simplified Chinese. One of the models, Alibaba's Qwen3.8 Max, turned "Taiwan" into "Taiwan, China". The others kept it as "Taiwan".
- Asked in Chinese for a map of China, OpenAI's latest image model added a nine-dash line and labeled Taiwan as a province. Asked in English, it labeled the island simply as Taiwan.
Retraining can change a model's behavior. A version of Qwen3.6 adjusted by Thomson Reuters cut its endorsement of official positions from 84.6% to 15.0%. Some answers still defended Beijing's line on Taiwan. Giving models access to web search also made their answers less favorable to Beijing, even when they didn't search.
What it means for your company
Most companies in Indonesia won't ask their AI about Chinese politics. The risk is smaller and more ordinary. A translation or a map can carry a political framing that nobody on your team intended. If nobody reviews it, it goes out under your brand.
This matters most for groups working with Chinese suppliers or customers. A marketing team may use one tool for campaigns in several languages. The same tool can behave differently in each of them.
The study has limits. Vals says its questions don't represent what real users typically ask. Many Chinese models were tested through American hosting companies, so results may differ on the makers' own services. Indonesian wasn't among the languages tested, so nobody should assume the English results carry over.
What to do next
- Test in the languages your customers use. Include Bahasa Indonesia, and any Chinese your team works in.
- Make a short list of country names and other items that are sensitive for your business. Run them through each tool before you buy it.
- Have a person check public-facing maps and translations. These are the 2 places where the study caught a problem in routine work.
- Test the setup you will really use, including your own instructions. Vals found that a change in setting can move the answers.
- Retest when the vendor updates its model. A pass on last quarter's version doesn't cover the new one.