The scene in the movie 3 Idiots, where Rancho asks the students and the head of the engineering college to define a non-existent term, feels strikingly relevant today amid the frenzy surrounding AI. In that scene, every student rushes to search for the made-up term in their textbooks without questioning the merit or genuine nature of the term itself. The frantic page-flipping and the desperate attempts to come up with an answer about something that doesn’t exist remind me of what’s happening among many CXOs today in their race to adopt Gen AI.

Every CXO is rushing to declare they have moved beyond the pilot stage and into monetization, claiming they have evolved from explorer to executioner, armed with a ready-made platform to deliver business impact. We have seen such madness and chaos before with every new wave of technology: Automation, AIOps, Cloud, VDI, and many more. So, is everything truly “green” in AI?

Recently, news circulated across the industry that Deloitte had refunded a sum to the Australian Government due to the errors in their output, apparently caused by AI. While some found this surprising, it wasn’t the first time something like this had happened. 

• In June 2024, McDonald’s stopped the use of its AI-powered drive-thru ordering system. 
• Replit, a coding assistant tool, accidentally deleted its actual database and created thousands of fictional users. 
• Air Canada was even compelled to compensate a passenger for the lies churned out by its chatbot. 

These are just a few instances of AI failures among the many that are happening around us. The fault doesn’t lie with the AI technology itself, but with the industry’s forecasters, analysts, and vendors.

We have heard these predictions many times before. When cloud computing emerged, industry analysts claimed that traditional IT infra services would vanish. Yet, even after 15 years, the cloud hasn’t replaced them. Around 2018-19, similar forecasts suggested that automation would eliminate half of all engineering jobs. Look where we are today.

Now, the same narrative is resurfacing. This time, it’s GenAI that’s expected to replace up to 50% of the IT workforce. Most vendors, particularly IT service providers, tend to echo these analyst predictions. But, can AI truly reduce the workforce significantly?

I remember the very first meeting I had with Professor Nandan Sudarsanam, IIT Madras, at his office. He clearly said that AI cannot guarantee perfect results. Rule-based programs will guarantee conformance to the rule, but AI models are benchmarked on predictive accuracy. At worst, they are just better than random chance; at best, we can get near-perfect predictions, so there is a wide range. This depends upon the quality of the data, availability of meaningful features and noise in the system, volume of data, complexity of the environment we seek to model, and data drift.

This means that if we deploy AI in a call center, up to 10% of customers may not get the right responses. Similarly, a code converter may not be able to convert all the legacy codes into a modern language, and a report generator may give unasked-for reports one out of 10 times. Are we ready to tolerate these margins of error?

Every CFO is asking for returns in the form of hard cash for the investment made on AI. The true quantitative returns can come from reducing the human workforce only, as AI cannot be expected to replace anything else, neither the hardware nor the software. If we have to replace humans with AI, the error percentage of the AI models has to be significantly low.

As of today, activities like writing, creating images, graphic design, and text analysis, are relatively more error-tolerant, with acceptable error rates of up to 50%. However, these activities alone cannot replace enough human effort to provide significant returns.

Some tasks like predicting an outage or a security threat in IT systems, forecasting demand for a specific item in the market, or recommending a loan to a customer, can tolerate a higher error margin (1% to 10%) because the consequences of an error are relatively minor. However, even here, the savings in the human workforce alone are often insufficient to generate a significant ROI.

The real savings will come from deploying AI in what I call ‘real-time business activities,‘ such as call centers, IT help desks, invoice, payment, claim, or order processing, fraud detection, and similar areas, where actions are executed and results get delivered in real time. These activities employ large human workforces across industries, however, AI must achieve extremely high accuracy in these tasks, typically producing fewer than 1% errors. 

But, how can we do that?

Well, there are many techniques to improve the accuracy (I mean reducing the errors), and one of them is Human-in-the-Loop (HITL). It means bringing humans to validate every outcome of AI models every single time. The moment we bring in HITL, the quality of the delivery improves, but not the savings. The CIO, CTO, or CISO will be happy with these qualitative improvements, but not the CFO who expects real cash.