Centuries ago, scholar Muhammad ibn Musa al-Khwarizmi popularized calculation using Hindu numerals, giving rise to the terms “algorithm” and “algebra.” The etymology offers a timely reminder: an algorithm is not an oracle, but a method for processing inputs into outputs. Today, artificial intelligence has rapidly evolved from predicting patterns to generating complex answers. However, as systems propose original proofs, legal briefs, and medical diagnoses, a critical gap has emerged: while the cost of producing plausible answers has plummeted, the cost of verifying their correctness has soared.
Mathematics illustrates this bottleneck clearly. Fields Medalist Jacob Tsimerman recently warned that AI will soon become superhuman at mathematics, while Terence Tao pointed out a growing crisis of “proof indigestion”—an overflow of complex proofs arriving inside institutions equipped only for proof scarcity. A single AI-generated output can take months or teams of experts to verify, exposing a fundamental issue in the economics of expertise. AI accelerates production without making verification, explanation, or oversight any cheaper.
This dynamic poses serious risks across critical industries:
-
Legal Systems: AI-hallucinated court citations continue to rise sharply worldwide, threatening legal precedents and individual liberties.
-
Software Development: Studies show developers using AI code assistants can actually work slower overall due to the increased burden of checking and debugging generated code.
-
Healthcare: Over-reliance on AI diagnostic tools has shown early signs of clinical deskilling, with doctors becoming less accurate at spotting conditions independently.
The challenge is especially acute in India, where AI tooling is being integrated at population scale into public infrastructure like UPI payments, digital identity systems, and rural healthcare diagnostics. While India excels at building digital rails and indigenous AI capability, independent evaluation capacity has lagged behind.
To prevent systemic risks, India must urgently establish a robust, independent verification framework:
-
Empower Safety Bodies: Properly fund and protect the independence of the IndiaAI Safety Institute, giving it full authority to audit models pre- and post-launch.
-
Contextual Benchmarking: Develop localized testing standards that evaluate AI performance across India’s diverse languages, dialects, and deployment conditions rather than relying solely on global, English-centric benchmarks.
-
Professional AI Auditing: Formalize AI auditing as an officially recognized profession to systematically evaluate privacy risks, algorithmic bias, and security across finance, healthcare, and public services.
Building machines to produce answers was only the first step. The next imperative is building the independent oversight required to know which answers to trust.
