Develop AI more easily with autoML technology

The result of all these human decisions is that complex models end up being "designed intuitively" rather than systematically, said Frank Hutter, head of the Machine Learning Laboratory at the University of Freiburg in Germany.

#A growing field called automated machine learning (autoML) aims to eliminate this guesswork. The idea is to let algorithms take over the decisions researchers currently have to make when designing models. Ultimately, these technologies could make machine learning more accessible.

Although automatic machine learning has been around for nearly a decade, researchers are still working to improve it. A new conference taking place in Baltimore today showcases efforts to improve the accuracy and streamline the performance of autoML.

There is strong interest in autoML’s potential to simplify machine learning. Companies like Amazon and Google already offer low-code machine learning tools that leverage autoML technology. If these techniques become more effective, it could speed up research and make machine learning available to more people.

The idea is to allow people to choose the question they want to ask, point the autoML tool at it, and get the results they want.

This vision is the "holy grail of computer science," said Lars Kotthoff, assistant professor of computer science at the University of Wyoming and conference organizer. "You specify the problem, and the computer knows how to solve it, and that's what you do. Everything." But first, researchers must figure out how to make these techniques more time- and energy-efficient.

What can automated machine learning solve?

At first glance, the concept of autoML may seem redundant—after all, machine learning is already about automating the process of deriving insights from data. But because autoML algorithms operate at a level of abstraction above the underlying machine learning models, relying solely on the output of those models for guidance, they save time and computational effort.

Researchers can apply autoML technology to pre-trained models to gain new insights without wasting computing power duplicating existing research.

For example, Mehdi Bahrami, a research scientist at Fujitsu Research Institute in the United States, and his co-authors presented recent work on how to use the BERT-sort algorithm with different pre-trained models to adapt to new purposes.

BERT-sort is an algorithm that finds what is called "semantic order" when training on a data set. For example, given movie review data, it knows that "great" movies are ranked higher than "good" and "bad" movies.

With the help of autoML technology, the learned semantic order can also be generalized to classify cancer diagnosis and even foreign language text, thereby reducing time and calculation amount.

"BERT requires months of computation and is very expensive, like $1 million to generate the model and repeat the process," Bahrami said. "So if everyone wants to do the same thing, That's expensive — it's not energy efficient, and it's not sustainable for the world."

Despite the promise shown in the field, researchers are still looking for ways to make autoML technology more computationally efficient. For example, with methods like Neural Architecture Search (NAS), where many different models are built and tested to find the best fit, the energy required to complete all these iterations can be significant.

Automatic machine learning can also be applied to machine learning algorithms that do not involve neural networks, such as creating random decision forests or support vector machines to classify data. Research in these areas is ongoing, and there are already many coding libraries available for people who want to integrate autoML technology into their projects.

Hutter said the next step is to use autoML to quantify uncertainty and address issues of trustworthiness and fairness in the algorithm. In this vision, criteria for trustworthiness and fairness would be similar to any other machine learning constraints, such as accuracy. AutoML can capture and automatically correct biases found in these algorithms before they are released.

Continuing progress in neural architecture search

But for applications like deep learning, autoML still has a long way to go. The data used to train deep learning models, such as images, documents, and recorded speech, is often dense and complex. It requires huge computing power to process. The cost and time to train these models can be prohibitive for anyone except researchers working at large corporations with deep pockets.

A competition at the conference requires participants to develop energy-efficient alternative algorithms for neural architecture search. This is quite a challenge because this technology is notorious for its computational requirements. It automatically cycles through countless deep learning models to help researchers choose the right one for their application, but the process can take months and cost more than a million dollars.

The goal of these alternative algorithms, known as zero-cost neural architecture search agents, is to make neural architecture search more accessible and greener by drastically slashing its computational requirements. The results take seconds to run, not months. Currently, these techniques are still in the early stages of development and often unreliable, but machine learning researchers predict they have the potential to make the model selection process more efficient.

The above is the detailed content of Develop AI more easily with autoML technology. For more information, please follow other related articles on the PHP Chinese website!