The Supreme Court of India on Tuesday flagged an alarming trend that has taken over the nation's top courts: Lawyers are using AI to draft petitions, leading to the submission of fabricated case laws and judicial quotes that are non-existent.
A three-judge bench comprising Chief Justice of India Surya Kant, Justice BV Nagarathna and Justice Joymalya Bagchi has issued a sharp statement to the bar, warning that the unchecked use of AI tools could threaten the integrity of the judicial process.
“We have been alarmingly told that some lawyers have started using AI for drafting,” Kant said during the hearing.
The bench highlighted that there were multiple instances in which AI 'hallucinations' making its way into official court pleadings. Nagarathna pointed to one such example where a lawyer cited a fictitious ruling.
“There was a case of Mercy vs Mankind which does not exist,” Nagarathna said.
These AI hallucinations are not only restricted to fake cases. Even when lawyers have referenced legitimate Supreme Court rulings, AI has generated false excerpts. Nagarathna noted that in some petitions citing real cases, the quoted portions never existed in the first place.
Kant added that similar fabrications recently appeared in a case before Justice Dipankar Datta, where all the legal precedents cited by the counsel were discovered to be fake.
AI hallucinations are a phenomenon in which generative AI models confidently invent false information, as oftentimes a model's only true motive is to generate an answer, and not necessarily the truth.
While AI can streamline tedious legal research and drafting, its propensity to hallucinate poses severe ethical and professional risks, especially in the field of the judiciary.
ALSO READ: AI Adoption To Evolve IT Roles, Not Eliminate Them, Says MeitY Secretary
Essential Business Intelligence, Continuous LIVE TV, Sharp Market Insights, Practical Personal Finance Advice and Latest Stories — On NDTV Profit.