The difference between boosting and bootstrap

Bootstrap and boosting are several commonly used resampling methods in machine learning. Among them, the bootstrap resampling method is mainly used for the estimation of statistics, and the boosting method is mainly used for the combination of multiple sub-classifiers.

bootstrap: Resampling method for estimating statistics (Recommended learning: Python video tutorial)

The bootstrap method is to randomly select n sample points from the original training data set DD of size n to form a new training set. This selection process is repeated B times independently, and then the model statistics are calculated using these B data sets. Estimates (such as mean, variance, etc.). Since the size of the original data set is n, there will inevitably be duplicate samples in these B new training sets.

The estimated value of the statistic is defined as the average of the estimated values ​​θbθb on the independent B training sets:

##boosting:

boosting trains k sub-classifiers in sequence, and the final classification result is determined by voting by these sub-classifiers.

First, randomly select n1n1 samples from the original training data set of size n to train the first classifier, denoted as C1C1, and then construct the training set D2D2 of the second classifier C2C2. Requirements: D2D2 Half of the samples can be correctly classified by C1C1, while the other half of the samples are misclassified by C1C1.

Then continue to construct the training set D3D3 of the third classifier C3C3. The requirements are: C1C1 and C2C2 have different classification results for the samples in D3D3. The remaining sub-classifiers are trained along similar lines.

The main principle of boosting in constructing a new training set is to use the most informative samples.

For more Python-related technical articles, please visit the

Python Tutorial column to learn!

The above is the detailed content of The difference between boosting and bootstrap. For more information, please follow other related articles on the PHP Chinese website!