What is hindering the progress of artificial intelligence? Or is it a data problem?

A new survey conducted by Forrester Consulting on behalf of Capital One shows that a lack of solid data foundations and solid data workflows is hindering enterprises from achieving better results in machine learning and artificial intelligence. Big progress.

According to a new report recently released by Capital One, "Achieving Key Business Outcomes with Actionable Machine Learning," although companies are increasingly integrating machine learning (ML) and artificial intelligence There has been some success getting (AI) into production, but they would have made greater progress if data management issues didn't get in the way.

The report is based in part on a Forrester survey of 150 data management decision-makers in North America in July this year, which found that 73% of decision-makers believe that the transparency, traceability and explainability of data flows are barriers to machine learning and AI. Key issues in application operationalization. The survey also found that 57% of respondents said internal silos between their data scientists and business operators hindered the deployment of machine learning.

David Kang, senior vice president and head of data analytics at Capital One, said: "We are still at a stage where machine learning algorithms themselves are not a barrier to people's success." "The key is data!"

When Capital One commissioned this survey, they thought the biggest challenge would focus on the actionability of machine learning. With the development of machine learning and artificial intelligence applications, MLOps (machine learning operations) has become an independent discipline, and it is also an area in which Capital One is investing.

But when this report came out, data decision-makers were most concerned about the lack of progress in building a solid data foundation, including data engineering and data infrastructure, Kang said.

"In some ways, this is disappointing. But in other ways, it is not surprising. Because leveraging data at scale requires a sustained focus on thinking and rethinking the data ecosystem Every capability in the system – how it is produced and consumed, how it is monitored, how it manages data in different ways. The transformation journey of the data ecosystem is still ongoing. It’s not something you do once and forget about. It’s Continued attention is needed."

Capital One's survey is similar to the findings of other recent studies. These studies found that data management issues slowed the pace and extent of adoption of machine learning and artificial intelligence. These include an MIT Technology Review report commissioned by Databricks in September that highlighted the dangers of improper data management on artificial intelligence; and an IDC study commissioned by Collibra in August that found , there is a correlation between companies with “data-intelligent” characteristics such as data cataloging, inheritance, quality management and governance, and market success.

If there is a common theme among these studies, it is that while the sophistication of existing machine learning and artificial intelligence technologies is growing rapidly, enterprises are finding that they have not done some core data management work well. , and these tasks are necessary to achieve these technological advances.

Businesses may find that ML or AI applications have a positive impact on a limited proof-of-concept (POC), but fail to take the necessary steps to ensure a smooth rollout into wider real-world production.

It may take a while before the technology you want to scale starts to make an impact in the market. The temptation is always there for these concepts to start seeing results and then suddenly find themselves somewhere with a bunch of data silos and a bunch of other data engineering infrastructure challenges.

Data science is still a fairly new discipline, and many companies are struggling to fill job openings. Capital One’s report found that 57% of respondents said they intend to use partnerships to fill gaps among data science practitioners. Kang said the lack of in-house expertise also makes it more critical for enterprises to establish core data infrastructure, making it easier for more advanced ML and AI use cases to be built on top of it and easier to repeat.

Capital One’s investigation also uncovered other issues slowing the adoption of machine learning and artificial intelligence. The company found that 36% of respondents cited "large, diverse, and confusing data sets" as a major obstacle, and 38% cited AI risks as the top challenge. 38% cited data silos across the organization and external data partners as a challenge to machine learning maturity.

The “hiccups” of data management don’t seem to be slowing down investment in artificial intelligence and machine learning (at least not yet). Capital One's survey found that 61% of decision makers plan to add new machine learning capabilities and applications in the next three years. More than half (53%) of respondents are currently prioritizing leveraging machine learning to improve business efficiency.

So, what are companies using machine learning for? Another interesting tidbit from the survey is that automated anomaly detection is the top use case for machine learning, with 40% of respondents reporting it as their top use case. This resonated with Kang, who helped Capital One build a machine learning-based anomaly detection system.

Other top use cases for ML and AI include: automated application and infrastructure updates (39%), and meeting new regulatory and privacy requirements for responsible and ethical AI (39%).

The above is the detailed content of What is hindering the progress of artificial intelligence? Or is it a data problem?. For more information, please follow other related articles on the PHP Chinese website!