A brief analysis of algorithms and models

The prerequisite for discussing issues is the clarification of concepts. Confusion of concepts is a major obstacle in R&D communication, which greatly affects the efficiency of communication. In the field of artificial intelligence, especially machine learning, algorithms and models are two related but different concepts. Understanding the meaning and connection between the two can make the issues we discuss more clear.

What is an algorithm?

In the computer field, algorithms have a clear definition. For the sake of simplicity, here is Baidu Encyclopedia’s description of the algorithm:

Algorithm refers to an accurate and complete description of the problem-solving solution. It is a series of clear instructions for solving the problem. The algorithm represents the use of A systematic approach describes strategic mechanisms for solving problems.

That is to say, for the algorithm, the required output can be obtained within a limited time for a certain standard input. Instructions in an algorithm describe a computation that, when run, can start from an initial state and a (possibly empty) initial input, go through a limited and clearly defined series of states, and finally produce an output and stop at a final state.

Different algorithms may use different time, space or efficiency to complete the same task. The quality of an algorithm can be measured by space complexity and time complexity.

What is a model?

Model is a word with a very broad conceptual extension. Here we also quote the description in Baidu Encyclopedia:

Through subjective consciousness with the help of physical or virtual representation, it constitutes an objective explanation of morphological structure. An object that expresses a purpose (objects are not equal to objects, not limited to physical and virtual, not limited to plane and three-dimensional).

A model can be a model toy, a scientific model, an economic model, a logical model, etc. For example, a globe is a model of the earth, rational man economic man is an economic model of human behavior, and a computer The layer 7 protocol in the network is the model of network protocols and so on.

Among them, scientific model is a general term for a type of research method in scientific research. It uses mathematical formulas, computer simulations or simple diagrams to represent a simplified nature. By analyzing this model, we hope to further understand Science, including explanation, testing of hypotheses, or analysis of data. Depending on different needs, scientific models can help understand phenomena through conceptual models, operational models can provide operational definitions, mathematical models can help with quantification, and image models can be used to visualize abstract concepts.

What needs to be focused on is the mathematical model. A mathematical model is an abstract, simplified structure about part of the real world and for a special purpose. It uses letters, numbers and other mathematical symbols to establish equations or inequalities, as well as charts, images, block diagrams, etc. to describe objective things. Characteristics and mathematical structural expressions of their internal relationships.

Simply put, a model is an abstraction of things.

Algorithms and models in machine learning

In the specific field of machine learning, algorithms and models also have more specific definitions.

"Algorithm" in machine learning

The algorithm in machine learning refers to a specific calculation method, that is, how to find the global optimal solution and make The process is efficient and accurate, essentially a computer algorithm.

Machine learning algorithms perform "pattern recognition," "learning" from data, or "fitting" to a data set. Machine learning algorithms can be described in pseudocode and implemented in any modern programming language, as can the analysis and description of algorithm efficiency. Multiple machine learning algorithms can be implemented together and provided in libraries with standard application programming interfaces (APIs). A popular example is the scikit-learn library, which provides implementations of many machine learning algorithms such as classification, regression, and clustering in Python.

The "model" in machine learning

The "model" in machine learning is the output of the machine learning algorithm running on the data set, which represents the output of the machine learning algorithm. What is learned is the rules, numbers, and any other algorithm-specific data structures used to make predictions. In other words, a machine learning model is a program automatically written, created, or learned by a machine learning algorithm to solve a specific type of problem. In a sense, the data structure can be regarded as a model. The following is a comparison between BTree and neural network models.

#The main issue to consider in machine learning is what kind of model to learn. For example, in a supervised learning process, the model is the conditional probability distribution or decision function to be learned. The sample space composed of data may contain various conditional probability distributions or decision functions, each of which corresponds to a model, so there are multiple models in this sample space.

Another concept related to models in machine learning is strategy. The strategy is the criterion for selecting the model with optimal parameters from the sample space. The smaller the error (loss function) between the model's classification or prediction results and the actual situation, the better the model. In other words, the strategy is to measure the quality of the model by introducing a loss function.

The relationship between algorithms and models in machine learning

Algorithms and models are often inseparable. An algorithm is the answer to a type of problem, a link in the model building process, and a link that gives the model the ability to "think". The model is the mathematical expression of a series of algorithms, including the concepts of data and tasks. Therefore, we evaluate the quality of a model rather than the quality of an algorithm. An algorithm only has its applicable environment and scenarios.

As far as machine learning is concerned, in layman's terms, the model can be approximately understood as an objective function, and the algorithm is a method for solving the objective function. All problems can have multiple modeling methods. For example, classification problems can be done with SVM or NB, but their objective functions are different. For example, in a decision tree, the model can be a binary tree model. If it is written as an objective function, it is a piecewise function. With this objective function, how to solve it? A typical algorithm is a greedy algorithm, or a heuristic algorithm, which starts from the root and adopts a certain splitting principle to split. The splitting principle is to maximize the information gain and minimize the variance of the Gini coefficient.

Decision tree-based solutions and their evolutionary variants, so far, have basically used heuristic splitting. Whether it is subsequent bagging, boosting, random forest, or deep forest, the algorithm is essentially the same. Nothing has changed, just the model used is changing.

Summary

Algorithms and models have broad general significance. Specifically, in the field of machine learning, an algorithm is a process implemented in code and run on data. The model is output by the algorithm and consists of model data and prediction algorithm. Machine learning algorithms provide a type of automated programming in which machine learning models represent programs. Once the concept is clear, there will be a basis for further study and discussion.

The above is the detailed content of A brief analysis of algorithms and models. For more information, please follow other related articles on the PHP Chinese website!