The truth about artificial intelligence and ROI: Can artificial intelligence really be achieved?

Today, more than ever, organizations are trusting and investing in the potential of artificial intelligence (AI) and machine learning (ML).

According to the 2022 IBM Global Artificial Intelligence Adoption Index, 35% of enterprises reported currently using artificial intelligence in their operations, and an additional 42% said they were exploring artificial intelligence. Meanwhile, a McKinsey survey found that 56% of respondents said they had adopted AI in at least one function in 2021, up from 50% in 2020.

But, can investments in AI deliver real ROI that directly impacts a business’s bottom line?

According to Domino Data Lab’s recent REVElate survey, which surveyed participants at the Rev3 conference in New York City in May. Attendees were surveyed, and many respondents seemed to think so. In fact, nearly half expect double-digit growth from data science. Nearly four-fifths of respondents (79%) said data science, machine learning and artificial intelligence will be critical to their company's overall future growth, with 36% citing it as the most critical factor.

Of course, implementing artificial intelligence is not easy. Other survey data show another side of strong confidence. For example, recent survey data from artificial intelligence engineering firm CognitiveScale found that while executives know that data quality and deployment are critical success factors in driving successful application development for digital transformation, more than 76% of executives are unsure how to implement successful application development in 12-18 Achieve the goal within a few months. Additionally, 32% of executives said it is taking longer than expected to bring AI systems into production.

AI Must Be Accountable

Bob Picciano, CEO of Cognitive Scale, told the media that ROI from artificial intelligence is possible, but it must be accurately measured according to business goals Description and personalization.

"If the business goal is to use historical data to make longer-term forecasts and improve forecast accuracy, then AI has a role to play," he said. "But AI must drive business efficiency responsibly - It’s not enough for an ML model to be 98% accurate.”

Instead, the ROI could be, for example, to improve call center efficiency, with AI-driven features ensuring a reduction in average call handling time.

“This ROI is what they talk about in the C-suite,” he explains. “They don’t talk about whether the model is accurate, robust or drifting.”

Shay Sabhikhi, co-founder and chief operating officer of Cognitive Scale, added that 76% of respondents said it was difficult to expand their efforts in artificial intelligence. Engaged, he wasn't surprised. "That's exactly what we're hearing from our enterprise customers," he said. One of the problems, he explained, is friction between data science teams and other organizations that don’t know what to do with the models they develop.

He said: “These models may have the best algorithms and precision recall, but they are shelved because they are essentially thrown to the development team and then they have to hurriedly put the application together. ”

At this point, however, organizations must take responsibility for their investments in AI because AI is no longer a series of scientific experiments, Picciano noted. “We call it going from lab to life,” he said. “I was at a chief data analytics officer meeting, and they were all asking, how do I scale? How do I industrialize AI?”

Is ROI the right metric for AI?

However, not everyone agrees that ROI is the best way to measure whether AI is driving value in an organization. Nicola Morini Bianzino, global chief technology officer at EY, said that measuring AI and businesses by "use cases" and then measuring it by ROI is the way to treat AI. Wrong way.

“To me, AI is a set of technologies that can be deployed almost anywhere in the enterprise—without isolating use cases from associated ROI analysis,” he said.

Instead, he explained, organizations simply need to use AI everywhere. "It's almost like cloud computing, two or three years ago, I had a lot of conversations with customers where they asked, 'What is the ROI? What is my business case for moving to the cloud? Now, post-pandemic, that conversation It doesn’t happen anymore. Everyone says, ‘I have to do this.’” Additionally, Bianzino noted that discussing AI and ROI depends on what you mean by “using AI.”

"Suppose you try to apply some self-driving capabilities - that is, computer vision is a branch of artificial intelligence," he said. "Is this a business case? No, because you can't do it without artificial intelligence Autonomous driving.” The same goes for companies like EY, which absorbs vast amounts of data and makes recommendations to clients — all without artificial intelligence. "It's something you can't separate from the process - it's intrinsic," he said.

Furthermore, by definition, AI is not productive or efficient on day one. Obtaining data, training the model, evolving the model, and scaling the model all take time. “It’s not that one day you can say, I’m done with artificial intelligence, and 100 percent of the value is there — no, it’s an ongoing capability that will get better over time,” he said. There’s no real end point in terms of the value that can be generated.”

Bianzino said that to some extent, artificial intelligence is becoming part of the cost of business. “If you’re in an industry that involves data analysis, you can’t not have AI capabilities,” he explains. "Can you isolate the business case for these models? It's hard and I don't think it's necessary. To me, it's almost the infrastructure cost of running a business."

ROI OF AI Hard to measure

Kjell Carlsson, head of data science strategy and advocacy at Domino Data Lab, an enterprise MLops provider, says that ultimately, what enterprises want is to measure the business impact of ROI—how much it contributes. But one problem is that this can be completely disconnected from the work that went into developing the model.

"So if you create a pattern that increases click-through rates by one percentage point, you're adding millions of dollars in profit to the business," he said. "But you can also create a good forecast sexual maintenance model, helping to provide advance warning before a machine needs repair." In this case, the monetary value can have completely different impacts on the organization, "even if one of them may end up being a more difficult problem," he added .

In general, organizations do need a “balanced scorecard” to track AI production. "Because if you're not putting anything into production, then that might be a sign that you've got a problem," he said. "On the other hand, if you put too much into production, that could also indicate a problem."

For example, the more models a data science team deploys, the more models they need to manage and maintain. "So you deployed so many models last year that you couldn't actually afford these other high-value models," he explained.

But another problem with measuring AI ROI is that for many data science projects, the result is not a model that goes into production. "If you want to do a quantitative profit-and-loss analysis of last year's trading, you probably need to do a rigorous statistical investigation of that," he said. “But without a model going into production, you’re leveraging AI to get the insights you gain along the way.”

Must Track Data Science Activities

Still, if you don’t track data science activities, organizations cannot measure the impact of AI. "One of the problems right now is that there is very little data science activity actually collecting and analyzing it," Carlsson said. "If you ask people, they'll say they really don't know how well their models perform, or how many projects they have, or how many CodeCommits your data scientists completed last week."

One of the reasons is data Scientists need to use very irrelevant tools. "This is one of the reasons why Git is growing in popularity as a repository, the single source of truth for data scientists in an organization," he explains. MLops tools like Domino Data Lab provide a platform to support these different tools. He said: "The extent to which organizations can create these more centralized platforms... is important. The consequences of artificial intelligence are what people care about most

Wallaroo CEO and Founder Vid Jain, who worked in high-frequency trading at Merrill Lynch for nearly a decade, said that at Merrill Lynch, his role was to deploy machine learning at scale and do it with a positive ROI.

The real challenge is not developing data science, cleaning data, or building transactional repositories (now called data lakes). The biggest challenge by far, he said, is taking these models, making them operational, and delivering business value.

He said: “Achieving ROI is very difficult – 90% of AI projects generate no ROI, or they do not generate enough ROI to make the investment worthwhile. “But that’s at the forefront of everyone’s mind. The answer is not the same thing. "

A fundamental problem, he explained, is that many people believe that operating on machine learning is not that different from operating on a standard application. There is a big difference, he added, Because AI is not static.

He said: “It’s almost like tending a farm because the data is alive and the data changes and you’re not done yet. “It’s not like you build a recommendation algorithm and then people’s buying behavior is frozen in time. People have changed the way they buy. Suddenly, your competitor has a promotion. Consumers stop buying from you. They turned to their competitors. You have to maintain it regularly.

Ultimately, each organization needs to decide how to align its culture with the ultimate goal of achieving AI. "Then you really have to empower people to drive that transformation, and then let those People who are critical to your existing lines of business feel they will get some value from AI. ”

He added that most companies are still in their infancy. "I don't think most companies are there yet, but I've definitely seen a shift in the last six to nine months where people are starting to take business results and business value seriously."

Artificial Intelligence ROI Remains Elusive

But how to measure the ROI of AI remains an elusive question for many organizations. "For some companies, there are fundamental issues, like they can't even get their model into production, or they can but they're blind, or they succeed but now they want to scale," Jain said . "But in terms of ROI, machine learning often doesn't have an associated profit or loss."

He explained that AI initiatives are often part of a center of excellence, and ROI is owned by the business units, while in other cases it is It's hard to measure.

“The question is, is AI part of the business? Or is it a utility? If you are a digital native, AI may be part of the fuel that runs the business,” he said. "But in a large organization that has a legacy business or is undergoing transformation, how to measure return on investment is a fundamental question they have to solve."

The above is the detailed content of The truth about artificial intelligence and ROI: Can artificial intelligence really be achieved?. For more information, please follow other related articles on the PHP Chinese website!