Detailed explanation of K-means++ algorithm in Python

K-means algorithm is a common unsupervised learning algorithm used to cluster data into different categories. The K-means algorithm is an improved version of the K-means algorithm, aiming to improve the efficiency and accuracy of initial cluster center selection. This article will introduce in detail the principle, code implementation and application of the K-means algorithm in Python.

1. Overview of K-means algorithm

K-means algorithm is an iterative algorithm. The process of each iteration is: first randomly select K initial clustering centers, Then each data point is assigned to the category of the initial cluster center closest to it, and then the centers of all clusters are recalculated and the cluster centers are updated. Repeat the above process until the convergence conditions are met.

K-means algorithm process:

1. Randomly select K data points from the data as the initial clustering center.
2. Assign data points to the cluster with the nearest cluster center.
3. Recalculate the center of each cluster.
4. Repeat 2-3 until the convergence conditions are met (the cluster center no longer changes, the maximum number of iterations is reached, etc.).
5. K-means algorithm steps

The K-means algorithm is an improved version of the K-means algorithm, which is mainly optimized in the selection of the initial clustering center. The initial cluster center selection steps of the K-means algorithm are as follows:

1. Randomly select a data point as the first cluster center.
2. For each data point, calculate its distance D(x) from the nearest cluster center.
3. Randomly select a data point as the next clustering center. Make sure that the greater the distance between this point and the existing clustering center, the greater the probability of being selected:
   a. Calculate each data The nearest distance D(x)^2 between the point and the existing cluster center.
   b. Calculate the sum Sum(D(x)^2) of all D(x)^2.
   c. Weight each data point according to the proportion of its nearest distance to the existing cluster center, and the probability is D(x)^2/Sum(D(x)^2).
   d. From sampling according to the above probability, select a data point as the next cluster center.
4. Repeat step 3 until K cluster centers are selected.
5. Python implements the K-means algorithm

Next, we will implement the K-means algorithm through Python.

First, import the necessary libraries:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans

Next, we generate a set of data for clustering:

n_samples = 1500
random_state = 170
X, y = make_blobs(n_samples=n_samples, random_state=random_state)

Then, we train through the KMeans module of sklearn K-means model:

kmeans = KMeans(init="k-means++", n_clusters=3, n_init=10)
kmeans.fit(X)

Finally, we visualize the clustering results:

plt.figure(figsize=(12, 12))
h = 0.02
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.imshow(Z, interpolation="nearest",
          extent=(xx.min(), xx.max(), yy.min(), yy.max()),
          cmap=plt.cm.Pastel1, aspect="auto", origin="lower")
plt.scatter(X[:, 0], X[:, 1], s=30, c=kmeans.labels_, cmap=plt.cm.Paired)
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1],
            marker="^", s=100, linewidths=3,
            color='black', zorder=10)
plt.title("K-means++ clustering")
plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)
plt.show()

4. K-means algorithm application scenario

K-means algorithm Suitable for data clustering problems without label information. Compared with the K-means algorithm, in order to ensure the rationality and uniqueness of the initial clustering center, the K-means algorithm is more suitable for situations where there is a lot of data or the data distribution is relatively scattered.

K-means algorithm can be used in data mining, image processing, natural language processing and other fields. Clustering algorithms can be used to find samples with higher similarity, which is also very useful for the visualization of big data.

In short, the K-means algorithm has good application prospects in data mining, cluster analysis, image recognition, natural language processing and other fields.

The above is the detailed content of Detailed explanation of K-means++ algorithm in Python. For more information, please follow other related articles on the PHP Chinese website!