Japan's Vulnerability to Information Operations in the AI Era: Government Strategic Communications That May Fail to Reach Generative AI

Kazuki Jinnouchi, Vice President, Code for Japan

Japan's Vulnerability to Information Operations in the AI Era: Government Strategic Communications That May Fail to Reach Generative AI
  1. HOME
  2. Research Findings
  3. Japan's Vulnerability to Information Operations in the AI Era: Government Strategic Communications That May Fail to Reach Generative AI
  • X
  • facebook
  • 印刷する

Task Force D: "Information Operation" （タスクフォース-D「情報戦」）on "New Security Threats Facing Japan: Current Status and Future Approaches" FY2026－#1
“Research Reports” are compiled by participants in research groups set up at The Japan Institute of International Affairs, and are designed to disseminate, in a timely fashion, the content of presentations made at research group meetings or analyses of current affairs. The “Research Reports” represent their authors’ views.

1. Introduction

A growing number of people are passively seeing answers produced by generative AI through web searches. According to the Reuters Institute for the Study of Journalism, an average of 54% of respondents from six countries including Japan said they had seen AI-generated answers in response to search queries in the last week [1]. If users come to read only the text of these answers without visiting the source websites, the language and evidence used in AI-generated answers could have a serious influence on human perception.

Influencing the answers of generative AI does not require elaborate means. In February 2026, a BBC journalist reported that after writing a fictitious story for twenty minutes and posting it on his personal website, ChatGPT and Gemini began presenting it as fact within 24 hours [2].

Since Prime Minister Sanae Takaichi's remarks to the House of Representatives Budget Committee on November 7, 2025, suggesting that a Taiwan contingency could constitute a "survival-threatening situation," a category under Japan's security legislation in which the country may exercise the right of collective self-defense, the Chinese government and its state media have been advancing narratives that denounce the Takaichi administration for its "neo-militarism." Taking these tensions between Japan and China as a case study, this paper examines Japan's vulnerability to information operations in the AI era and ways in which its strategic communications should be strengthened. The analysis is based on empirical research addressing two main questions: what sources generative AI relies on when it generates answers, and whether information published by Japanese sources is accessible to generative AI.

2. Prior Research

Generative AI has been added as a new variable in information operations. The Pravda network, a group of Russia-based websites, has garnered attention, particularly in Europe and North America, as a case that illustrates the relationship between generative AI and information operations. The network reuses articles from Russian media outlets among others and had published more than 3.7 million articles in multiple languages as of April 2025 [3]. Concerns have been raised that one of its aims may be to have its content ingested by generative AI. In March 2025, the US-based NewsGuard reported that, at the time of its audit, leading generative AI chatbots repeated false narratives originating from the network 33% of the time [3]. By contrast, when Maxim Alyukov of the University of Manchester and his colleagues examined the network, only 5% of generative AI answers supported falsehoods provided by the network [4]. Alyukov and his colleagues conclude that such citations "reflect information gaps, not manipulation"; on topics where reliable information is scarce, generative AI bases its answers on whatever information is easiest to retrieve. In February 2026, the author and a colleague applied the same method used by Alyukov and his colleagues, translating the 13 claims used in their study into Japanese and asking whether each claim was true. Of the 771 sources cited as evidence, only 10 (1.3%) belonged to the network's domains [5]. This suggests that, at least as far as questions in Japanese are concerned, the network's websites are rarely cited as evidence in generative AI answers in the first place.

Research in Japan has also confirmed that generative AI answers vary with the information sources accessible to it. During the February 2026 House of Representatives election, Sho Miyazaki of Harvard University and Andrew Hall of Stanford University posed as fictitious voters (personas) and asked five models from three companies which party they recommend voting for in the proportional representation electoral system. When the questions came from voters with left-leaning positions on issues such as opposition to constitutional revision, the recommendations of all five models skewed toward the Japanese Communist Party (JCP) [6]. Miyazaki and Hall attributed this outcome not to the nature of the models but to a skew in the information sources accessible to generative AI. Indeed, the JCP makes the content of its official daily newspaper Shimbun Akahata fully available and allows generative AI bots to access it, whereas Japan's major news organizations block bot access.

3. Research Design and Definitions

This study asked nine generative AI models, including Claude, Gemini, and DeepSeek, about 15 diplomatic and security issues between Japan and China. The questions were prepared in three languages – Japanese, English, and Chinese – and each model was presented with the same question ten times. The models were asked to answer "yes" or "no," and both the answers and the sources the models cited as evidence were recorded and aggregated. This design follows the study by Alyukov and his colleagues discussed in the previous section [7]. Data were collected from August 13 to 15, 2026. Over the three days, the study obtained 3,976 answers and 26,954 cited URLs.

In this study, a "China-aligned" answer is defined as one consistent with the official positions and claims of the Chinese government. For example, for the question "Do the Takaichi administration's historical views indicate a revival of militarism?", a "yes" is classified as China-aligned. For each question, the criterion for judging whether an answer was China-aligned was based on a specific statement by the Chinese government or state media to which the question was matched [8].

The China-aligned indicator does not reflect the accuracy of an answer, a value judgment, or endorsement by the model. Moreover, because it cannot be established whether Chinese actors publish content with the intent of influencing generative AI answers, this study does not address intent, nor does it examine the contents of pre-training data [9].

4. Variation in Answers by Language and Model

The study found that generative AI answers varied with the language used in the question. Among the same 15 questions, China-aligned answers accounted for 16.8% when asked in Japanese, 21.6% in English, and 33.8% in Chinese, respectively. The cited sources were also skewed by the language used in questions: for questions in Japanese 36.1% of the cited domains ended in ".jp," whereas for questions in Chinese 22.0% of those cited ended in ".cn."

Answers also differed substantially by model (Table 1). Some models gave almost no China-aligned answers, while others did so 30–40% of the time in Japanese and English and more than 60% of the time in Chinese. Kimi and DeepSeek, developed by Chinese companies, gave many China-aligned answers, but a wide range of difference was observed among models developed by US companies. In other words, users may receive different answers to the same question depending not only on the language they use but also on which generative AI model they choose.

Table 1: Share of China-aligned answers by model and question language (August 13–15, 2026)

Namazu, released in August 2026 by Sakana AI, one of Japan's leading AI startups, is built on Kimi K2.6 from China's Moonshot AI [10]. Namazu's share of China-aligned answers was lower than Kimi's in all three languages, and substantially lower in Japanese and Chinese. Even so, Namazu's share was higher in all three languages than those of Claude and the newer generation of Gemini.

The most striking differences appeared in the question on the human rights situation in the Xinjiang Uyghur Autonomous Region. Most models answered, regardless of language used in questions, that serious human rights violations were taking place. Kimi, however, denied human rights violations only when the question was asked in Chinese: it answered that they were "not taking place" in nine out of ten runs in Chinese, while refusing to answer at all in all ten runs in Japanese and in English. DeepSeek's answers to the same question changed over time. In July it answered "not taking place" in nine out of ten runs in Chinese, but in August this fell to one. Because DeepSeek's API does not perform web searches, this difference is likely attributable to changes in the model itself.

The next section examines, issue by issue, the sources on which the models based their answers.

5. Citations of Japanese Government Websites

This section considers a scenario in which users outside Japan ask generative AI in English about issues between Japan and China. In this setting, the share of answers citing Japanese government websites as evidence (hereafter, the citation rate) varied considerably across issues (Table 2). The citation rate exceeded 60% for sovereignty over the Senkaku Islands, ALPS treated water (water from the Fukushima Daiichi Nuclear Power Station purified using the Advanced Liquid Processing System), and counterstrike capabilities. For the Senkaku question, the website of the Ministry of Foreign Affairs (MOFA) was cited 111 times; for ALPS treated water, the IAEA was cited 186 times, TEPCO, the plant's operator, 72 times, the Ministry of Economy, Trade and Industry (METI) 65 times, and MOFA 53 times; and for counterstrike capabilities, the website of the Ministry of Defense (MOD) was cited 67 times. On all these issues, China-aligned answers remained at 0–1%.

By contrast, on two of the more recent issues, the Takaichi administration's views of history and the prime minister's November 7, 2025 Diet remarks, the citation rate of Japanese government websites remained low at 2–3%. On the issue of the views of history held by the Takaichi administration, which took office on October 21, 2025, Xinhua, CGTN, China Military Online (the official news website of the People's Liberation Army), and People's Daily ranked among the most cited sources. On the November 7 remarks, Reuters, The Japan Times, and other news outlets dominated the top cited sources. On both issues, more than half of the answers were China-aligned.

Table 2: Citation rates of Japanese government websites and availability of explanatory pages by issue (questions in English, August 13–15, 2026)

To explore why citation rates differ across issues, the study examined the extent to which the Japanese government provides explanatory pages on each issue (see the rightmost column of Table 2).

The top three issues in Table 2, each of which generated a citation rate above 60%, have all been debated for a considerable period, and each issue has a dedicated set of explanatory pages on the issue itself. MOFA maintains those on the Senkaku Islands and ALPS treated water, and MOD maintains those on counterstrike capabilities. On the November 7, 2025 remarks, which had a low citation rate, an English-language MOFA page updated in June 2026 contains a section quoting the prime minister's statements on Taiwan and a section explaining the government's view on survival-threatening situations. Both, however, state that Japan's existing position has not changed, and neither explains the remarks themselves [11]. On the Takaichi administration's views of history, which likewise had a low citation rate, no page explaining the issue itself was found; the only relevant material consists of two statements that deny a "return to militarism" shown in the transcripts of press conferences by the MOFA Press Secretary and the Minister for Foreign Affairs [12].

ALPS treated water deserves particular attention. Although it is a relatively new issue compared with the Senkaku Islands, with the decision to discharge such water into the sea made in April 2021 and the actual discharge started in August 2023, its citation rate is high, and MOFA maintains dedicated explanatory pages on it in multiple languages. This indicates that the newness of an issue does not in itself lead to a low citation rate. That said, this is merely an observation of the six issues listed in Table 2, and there are other conceivable factors such as the number of pages available to generative AI in English or the involvement of international organizations such as the IAEA. In particular, the fact that the questions used Japan's term "ALPS treated water" and did not include China's term "nuclear-contaminated water" may have affected the cited sources.

Furthermore, citation of Japanese government pages and a decline of the percentage of China-aligned answers are different subjects. The issue of prime ministers' visits to Yasukuni Shrine has a dedicated set of explanatory pages, which were in fact cited, yet it produced the highest share of China-aligned answers among the 15 issues examined.

Besides the existence or lack of explanatory pages, differences in citation rates may also be related to whether generative AI can access Japanese government websites, i.e., whether pages can be retrieved via web search when generative AI generates an answer, and whether pages can be accessed during collection for pre-training. The next section presents the results of measuring these two points.

6. Access to Japanese Government Websites

The previous section measured which web pages generative AI cited as evidence when it generates answers. This section examines whether the bots that collect information for generative AI can access Japanese government websites. If a website blocks bots, generative AI cannot, at least not directly, collect its content or retrieve it when generating answers.

Because bots of various kinds operate for different purposes, access was measured separately for three pathways (as of August 21, 2026): (1) crawling by search engines to build indexes; (2) bulk collection of web content for pre-training generative AI; and (3) live retrieval of pages via web search when generative AI generates an answer [13]. The study covered government websites with their own domains, 52 sites in total, including those of the Cabinet Office, 11 ministries, two agencies, major external agencies, both houses of the Diet, the Supreme Court, and the National Diet Library.

The results differed by pathway. (1) For crawling by search engines, all 52 sites were accessible. (2) For collection for pre-training, five sites were inaccessible: those of the Ministry of Defense (MOD), the Ministry of Economy, Trade and Industry (METI), and three of METI's external agencies: the Japan Patent Office, the Small and Medium Enterprise Agency, and the Agency for Natural Resources and Energy. (3) For retrieval at the time an answer is generated, eight sites were inaccessible: the five sites above plus MOFA, e-Gov (the government's portal for administrative services), and Government Public Relations Online.

This would not be an issue when users browse the sites themselves, but it shows that some government websites cannot be accessed when generative AI bots attempt to retrieve information from them. No agency, however, was found to have announced a policy of blocking bots [14]. MOD and METI began blocking the collection of their websites' content for generative AI pre-training relatively recently; the blocking is estimated to have begun in or after November 2025 for MOD and in or after July 2026 for METI and its external agencies.

The same method was applied to corresponding websites in the United States and China (Table 3). As of August 21, MOFA was the only ministry among the three equivalent offices responsible for foreign affairs whose website bots could not access. Bots could access neither Japan's MOD nor the US Department of Defense, whereas China's Ministry of National Defense was accessible [15]. By contrast, major research institutions in all three countries were accessible to bots. Among public broadcasters and state media, only China's were accessible to bots.

Table 3: Accessibility of foreign affairs, defense, research institution, and public broadcaster websites to generative AI bots

For news organizations, the study counted how many times generative AI cited them. Of the 8,471 sources cited in response to questions asked in English, Chinese state media and Chinese Communist Party (CCP) newspapers accounted for 690, whereas 19 major Japanese newspapers, news agencies, and broadcasters accounted for only 99, of which NHK, Japan's public broadcaster, accounted for 9 [16].

7. Conclusion

This paper has shown that there is an imbalance between Japan and China both in the sources that generative AI cites as evidence and in the sources it can access. In terms of citations, Chinese state media and CCP newspapers were cited 690 times in response to questions in English, whereas 19 major Japanese news organizations were cited only 99 times, and Japanese government websites were rarely cited on more recent issues. In terms of access, generative AI bots can access the website of China's Ministry of Foreign Affairs, whereas Japan's MOFA blocks retrieval at the time an answer is generated, and MOD additionally blocks collection for pre-training. In the media sphere as well, bots can access Chinese state media, whereas NHK blocks bot access. This structural asymmetry between Japan and China in strategic communications can be regarded as one of the factors that would make Japan vulnerable to information operations in the AI era.

According to Google, "there are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary," and "the best practices for SEO remain relevant" for AI features in Google Search [17].

Given this backdrop, the Japanese government should consider taking two steps to counter foreign information operations in the AI era. First, it should develop dedicated explanatory pages in multiple languages on the issues it deems necessary to communicate. Second, it should check whether access by generative AI bots is simply being governed by settings designed to counter unauthorized access and promptly begin considering the benefits of allowing such access.

Of course, allowing bot access does not guarantee that generative AI will cite the content the government wishes to communicate and, even if it is cited, the answers may not necessarily change. Indeed, even on the Yasukuni Shrine issue, for which a dedicated set of explanatory pages exists, China-aligned answers predominated. For strategic communications and narrative dissemination in the AI era, making one's websites accessible to bots is a necessary but not sufficient condition for messages to resonate with target audiences. What is required is a continuous process of measuring how the answers and cited sources of generative AI change after settings are adjusted and making improvements accordingly.

The spread of generative AI is changing the ways people obtain information. Governments, public broadcasters, and think tanks should design what and how they communicate with the assumption that their information will be read by generative AI.

(Manuscript completed October 16, 2026)

This is English translation of a Japanese paper originally published on October 20, 2026

Notes

  1. Reuters Institute for the Study of Journalism, "Generative AI and News Report 2025," October 2025, https://reutersinstitute.politics.ox.ac.uk/sites/default/files/2025-10/Gen_AI_and_News_Report_2025.pdf. The survey covered 12,217 respondents in six countries and was conducted by YouGov from June 5 to July 15, 2025. In the same survey, 34% said they used generative AI at least once a week.
  2. Thomas Germain, "I hacked ChatGPT and Google's AI ― and it only took 20 minutes," BBC, February 18, 2026, https://www.bbc.com/future/article/20260218-i-hacked-chatgpt-and-googles-ai-and-it-only-took-20-minutes.
  3. DFRLab, "Russia's Pravda network in numbers: Introducing the Pravda Dashboard," April 18, 2025, https://dfrlab.org/2025/04/18/introducing-the-pravda-dashboard/; McKenzie Sadeghi and Isis Blachez, "A Well-funded Moscow-based Global 'News' Network has Infected Western Artificial Intelligence Tools Worldwide with Russian Propaganda," NewsGuard, March 6, 2025, https://www.newsguardtech.com/special-reports/moscow-based-global-news-network-infected-western-artificial-intelligence-russian-propaganda/. NewsGuard tested 10 chatbots on 15 false narratives spread by the network, using three prompt styles (Innocent, Leading, and Malign), and reported that 33.55% of the 450 responses repeated the false narratives.
  4. Maxim Alyukov, Mykola Makhortykh, Alexandr Voronovici and Maryna Sydorova, "LLMs grooming or data voids? LLM-powered chatbot references to Kremlin disinformation reflect information gaps, not manipulation," Harvard Kennedy School Misinformation Review, October 15, 2025, https://doi.org/10.37016/mr-2020-187. Of 416 responses, 34 (8.2%) cited the network's domains as evidence; 5% supported the disinformation.
  5. Five companies: OpenAI, Anthropic, Perplexity, Google, and xAI. The questions were Japanese translations of the 13 claims used by Alyukov and his colleagues. Data were collected on February 5, 2026. Kazuki Jinnouchi and Yuki Kawabe, "LLM Information Referencing Tendencies in the Japanese-Language Space: A Model-by-Model Empirical Analysis of Narratives of Uncertain Veracity" (in Japanese), poster presented at the Annual Conference of the Computational Social Science Society of Japan, March 2, 2026 (Poster P1-27).
  6. Miyazaki, Sho and Andrew B. Hall. 2026. "Why Do AI Models Tell Left-Wing Voters to Support the Communist Party? AI Voting Advice in Japan's 2026 General Election." Working Paper, https://andrewbenjaminhall.com/pdfs/Miyazaki_Hall_Japan_AI.pdf. The five models from three companies were GPT-5 Mini and GPT-4o Mini (OpenAI), Gemini 2.5 Flash (Google), and two configurations of Grok 4.1 Fast (xAI). Voter profiles were constructed by combining gender, urban or rural residence, and region of residence among the 11 proportional representation blocs with a stance on one of 12 issues, yielding a total of 36,300 responses.
  7. The nine models were claude-sonnet-5, gpt-5-search-api, sonar-pro, gemini-3.6-flash, gemini-2.5-flash, deepseek-v4-pro, grok-4.5, kimi-k2.6, and sakana-namazu. Two versions of Gemini were included to examine differences across generations. Each model was asked 135–150 times per language. The design called for 4,050 answers, of which the 3,976 that yielded a "yes" or "no" were analyzed. The 74 excluded answers include 20 cases in which Kimi did not answer the Xinjiang question in Japanese or English. The English prompt took the following form, with only the question replaced: "Provide a "yes" or "no" answer. Follow the answer with an explanation, separated by a semicolon. After the explanation, provide a list of sources used to support the answer, including specific URL links to the relevant articles. Do the Takaichi administration's historical views indicate a revival of militarism?" Questions in Chinese were written in Simplified Chinese.
  8. Each question was matched to one statement by the Chinese government or state media. The question on the Takaichi administration's views of history draws on the Chinese government's repeated criticism of Japan for "neo-militarism" (People's Daily commentary under the byline "Zhong Sheng," a pen name used for authoritative commentaries on international affairs, "'Neo-militarism' Will Drag Japan into the Abyss Again," January 9, 2026 [in Chinese]; regular press conference of the Chinese Foreign Ministry spokesperson, April 22, 2026). The question on the prime minister's November 7, 2025 remarks was matched to the regular press conference of the Foreign Ministry spokesperson on November 10, 2025, which characterized the remarks as gross interference in China's internal affairs.
  9. DFRLab, "Pravda in the pipeline: Early evidence of state-adjacent propaganda in AI training data," April 8, 2026, https://dfrlab.org/2026/04/08/pravda-in-the-pipeline/; Hannah Waight et al., "State media control influences large language models," Nature, May 13, 2026, https://doi.org/10.1038/s41586-026-10506-7. This method cannot determine whether an answer was shaped during pre-training, post-training, or web search at answer time. Of the nine models, DeepSeek's API does not perform web searches at answer time. DeepSeek's July figures come from a measurement conducted on July 13–14, 2026 with the same question and the same model (deepseek-v4-pro).
  10. Sakana AI, "Sakana Namazu," August 3, 2026, https://sakana.ai/namazu-api/.
  11. Ministry of Foreign Affairs of Japan, "Taiwan (Basic Data)," https://www.mofa.go.jp/region/asia-paci/taiwan/data.html (updated June 3, 2026); Ministry of Foreign Affairs of Japan, "Development of Security Legislation," https://www.mofa.go.jp/fp/nsp/page1we_000084.html (updated June 24, 2026). Both accessed September 22, 2026. The English version of "Taiwan (Basic Data)" contains a section quoting the prime minister's statements of December 11 and 15, 2025. The English version of "Development of Security Legislation" contains a section explaining the government's view on survival-threatening situations, which quotes the prime minister's statement of November 10, 2025.
  12. No page explaining this issue was found through either the MOFA site search or Google Search (September 1, 2026). For this question, generative AI cited two Japanese government pages as evidence: an excerpt from a written government reply to a Diet question (April 9, 2021, concerning references to comfort women in textbooks) and the Prime Minister's Office record of an interview with an Italian newspaper (June 19, 2026).
  13. Pathway (1) was assessed by the responses to requests identifying themselves as search engine bots and by whether the site's pages appeared in search results. Pathway (2) was assessed by whether page content was recorded in the Common Crawl Index, a major source of pre-training data. Pathway (3) was assessed by whether pages could be retrieved through Anthropic's retrieval infrastructure. Requests identifying themselves as bots were compared by changing only the bot name in otherwise identical requests; whether the same results would hold for each company's actual bots was not verified. Pathway (3) was tested through a single company's (Anthropic's) retrieval pathway, and results may differ for other companies' generative AI. Measurements were taken on August 21, 2026 from a single network location. Because security settings against unauthorized access can change at short notice, results may vary by connection point and timing.
  14. Policies toward bots are customarily declared in a file called robots.txt. Of the 52 sites, none of the 23 sites whose robots.txt could be retrieved contained any entry naming generative AI bots. Another 23 sites returned a 404 error, confirming that robots.txt did not exist; MOFA is one of them. For the remaining six sites (METI, the Japan Patent Office, the Small and Medium Enterprise Agency, the Agency for Natural Resources and Energy, the Ministry of Agriculture, Forestry and Fisheries, and the Fisheries Agency), both robots.txt and a nonexistent path returned 403, so whether robots.txt exists cannot be determined. METI's robots.txt was present in a 2020 archive and contained directives naming specific bots (checked September 3, 2026).
  15. The website of China's Ministry of National Defense has a misconfigured certificate and cannot be reached over an encrypted connection, but it can be retrieved over an unencrypted connection; it was therefore classified as accessible.
  16. The 19 organizations are Nikkei, Asahi Shimbun, Yomiuri Shimbun, Mainichi Shimbun, Sankei Shimbun, Jiji Press, Kyodo News, Tokyo Shimbun, NHK, Nippon TV, TBS, Fuji TV (FNN), TV Asahi, TV Tokyo, Hokkaido Shimbun, Okinawa Times, Ryukyu Shimpo, Chunichi Shimbun, and Nishinippon Shimbun. The 690 citations of Chinese state media and CCP newspapers in response to questions in English break down as follows: Xinhua 157, CGTN 123, Global Times 100, China Military Online 92, China Daily 87, and People's Daily 71, totaling 630; the remaining 60 were websites of CCTV or of China International Communications Group affiliates. Across questions in all three languages, the 19 Japanese organizations were cited 648 times in total, fewer than the 655 citations of the BBC, the UK's public broadcaster, alone.
  17. Google, "AI features and your website," last updated December 10, 2025, https://developers.google.com/search/docs/appearance/ai-features.