- A flawed AI-assisted intelligence report nearly led the US to intercept a Chinese vessel, CNN reported
- The intel report falsely claimed the ship carried nuclear weapon components in the Middle East
- US forces prepared to board the vessel before the AI error was discovered and the action was halted
A flawed intelligence assessment generated with the help of artificial intelligence reportedly brought the US military close to intercepting a Chinese vessel in the Middle East during the war with Iran, raising concerns over the risks of relying on AI for military decision-making.
CNN reported that a US military intelligence report circulated this spring claimed that a Chinese ship in the region was carrying components linked to a nuclear weapons programme. The assessment triggered preparations for an operation to intercept the vessel.
Two sources told CNN that armed US personnel were preparing to board the ship, while military aircraft were also reportedly airborne as the operation was being planned.
ALSO READ: China's Rare Earth Consolidation Could Hand State Firm Stake In US Defence Supplier: Report
The operation was halted only after officials took a closer look at the intelligence report and discovered that it had been produced with assistance from an AI chatbot. The chatbot had incorrectly identified the material the vessel was carrying, according to CNN.
One source described the intelligence assessment as entirely false and said it had almost started a war. According to CNN, the analyst, working with intelligence reporting on the ship's manifest originating from US Special Operations Command Pacific, queried a chatbot about the information.
The AI system reportedly combined open-source information with classified signals intelligence available in US government holdings before reaching the incorrect conclusion about the cargo.
The analyst then used AI again to turn the findings into a standard intelligence report, which was subsequently circulated among military officials.
The reported incident has renewed concerns about AI “hallucinations”, instances in which AI systems generate inaccurate or unsupported information, particularly when such outputs are used in military targeting.
ALSO READ: US Drafts Rules To Spare Pharma From Strictest China Investment Curbs: Report
A former senior US official familiar with AI systems used by military and intelligence analysts told CNN that internal tools are largely based on commercial AI technology.
The sources said the Chinese ship episode was not an isolated example of AI-generated errors within the intelligence community.
Some intelligence officials have expressed concern that the rapid adoption of AI could increase pressure on analysts to produce assessments faster, potentially increasing the risk of mistakes.
As one source put it, according to CNN: “AI allows you to get to a bad idea faster.”
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.