Why AI Keeps Security Experts Awake At Night?

Some of the earliest adopters of innovative tech aren't blue-chip companies but shady players who operate on the margins. Any tech aficionado will tell you how some of the pioneers of streaming, e-commerce and affiliate marketing were companies behind sleaze and porn. Frederick Lane, author of 'Obscene Profits: The Entrepreneurs of Pornography in the Cyber Age' put it best when he said that the porn industry blazed a commercial path that other industries hastened to follow. 

Given the billions of dollars that cybercrime pulls in nowadays, it shouldn't come as a surprise that cybercriminals—many of them state-sponsored threat actors—are using AI in innovative ways to fuel cybercrime sprees. Anthropic, the company behind Claude AI, which is preferred by some users for its excellence in natural language processing, just released a threat intelligence report that is eye-opening. In the report, the AI company has laid bare how cybercriminals used Claude with stunning effect, possibly boosting illicit earnings from crime manifold. By using Claude for evil, these hackers were able to build sophisticated malicious tools that they did not have the skills or expertise to otherwise develop.

Anthropic provided multiple use cases—from cybercriminals using Claude for developing malware tools to getting it to create scary ransom notes, to state-sponsored hackers using AI to create elaborate false identities, work repositories and more to bag jobs and stay employed at Fortune 500 companies as remote workers. It is clear from these cases that AI models are being used to run sophisticated criminal operations as cybercriminals embed AI in all aspects of their malicious operations. What is increasingly worrisome is that AI is lowering the barriers to sophisticated cybercrime as criminals with basic technical skills are leveraging AI to engage in activity that might have required years of training in the pre-AI era.

This is alarming news for CISOs leading security at companies since this makes cybersecurity increasingly tougher to manage, especially as AI tools are self-learning and have the ability to adapt in real-time and evade security systems deployed for malware detection. Cybersecurity strategy and the CISO role is more important than ever in the AI-led times we live and work in.

While AI being used for crime is a huge challenge, another is the one where some of us are forming deep, delusional and unhealthy attachments with conversational AI chatbots. We focused on this last week. Mustafa Suleyman, who leads Microsoft AI published a blog spotlighting these fears and explained that he doesn't see this problem limited to those who already fall into the high-risk zone when it comes to mental health. When industry leaders voice such fears aloud it's time to double down on ensuring the sector builds strong ethical guardrails.

Speaking of ethics, Perplexity AI has started an initiative where publishers will earn some revenue when their content is read on Perplexity’s Comet internet browser, appears in search queries or is used to complete tasks by Comet's AI. AI companies training their AI tools on publisher content and ultimately aiming to generate revenues need to not merely attribute results to the right sources but also pay for such use of content. However, judging by the litigation globally in this space, there is a long way to go before publishers get their due.

There has also been considerable coverage on OpenAI starting business operations in India, the first jobs listed at its India office and Sam Altman’s proposed visit. While the fundamentally unfair Trump tariffs may have resulted in a chill setting over Indo-US relations, businesses remain gung-ho over the massive opportunities India offers and want to be part of the India story. Incidentally, some may have missed it, but Kyndryl, the world’s largest provider of IT infrastructure services, has committed to spend a significant $2.25 billion in India over the next three years. A chunk of this will be used to set up an AI lab in India, while the company, which was spun off from IBM earlier this decade, will also set up operations in Tier 2 and Tier 3 cities across India. This proposed investment is a robust endorsement of India’s continuing growth potential.

Meanwhile, here are some of the other notable AI-related reads from the past few days:

• India's Compute Capacity To Surge: Yotta CEO Predicts 1 Lakh GPUs Within 18 Months

• How Observability Can Help AI Draw From Unified, Real-Time View

• Nvidia Forecasts Decelerating Growth After Two-Year AI Boom

• India Leads The World In AI-Driven Leadership Transformation: Microsoft Report

• NASA, IBM Develop 'Surya': Here's How Advanced AI Model Can Protect From Solar Storms

• AI Can Reshape Half Of Roles In Indian Banking: Report

• IBM To Drive AI-Powered Managed Services Transformation At Vi

• Indians Most Worried About AI Taking Over Jobs, Finds Survey

• Google Pixel 10 Series Launch: Tech Giant Shifts Focus To AI In Its Latest Smartphone Lineup

• Elon Musk's xAI Sues Apple, OpenAI For Alleged Anti-Competitive Practices

• 'Tongue-In-Cheek Name': Elon Musk Unveils AI-Driven Software Firm 'Macrohard' To Take On Rival Microsoft

Till next week, 

— Ivor Soans