This spring, at the height of the war with Iran, US forces came within moments of boarding a Chinese ship in the Middle East — all because a chatbot got its cargo wrong, as CNN’s Katie Bo Lillis and Zachary Cohen report. An intelligence report circulating across the military claimed the vessel was carrying components of a nuclear weapons program. Armed personnel prepared to board, and military planes were already in the air.
Just before the operation, officials discovered that a special operations command analyst had built the report with help from artificial intelligence. The chatbot had fused open-source intelligence with secret signals intelligence and misidentified the ship’s cargo. The analyst then used AI again to package the findings into a standard intelligence report and disseminated it.
The report was “entirely false,” one source told CNN — but it also “almost started a war.” Any US operation against a Chinese vessel risked spiraling into armed conflict with Beijing.
The near-miss comes as the Pentagon races to weave AI into everything from target selection to logistics. In January, Defense Secretary Pete Hegseth unveiled the department’s “Artificial Intelligence Acceleration Strategy,” vowing to “unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI.”
Related on the SLP
But officials describe a decentralized effort, with different parts of government using different tools under different safety standards — and no single standard for verifying what those tools produce. “The internal tools are mostly just copies of the commercial stuff wearing lipstick,” said a former senior US official familiar with the systems.
Meanwhile, AI-assisted targeting is ramping up with “no real guidance for how having a human in the loop will prevent civilian casualties or fratricide,” another source said. Some veteran officials also worry that younger analysts, native to these tools, trust them too uncritically. As one source put it: “AI allows you to get to a bad idea faster.”
