Anomaly detection problem based on time series

Anomaly detection problems based on time series require specific code examples

Time series data is data recorded in a certain order over time, such as stock prices, temperatures changes, traffic flow, etc. In practical applications, anomaly detection of time series data is of great significance. An outlier can be an extreme value that is inconsistent with normal data, noise, erroneous data, or an unexpected event in a specific situation. Anomaly detection can help us discover these anomalies and take appropriate measures.

There are many commonly used methods for anomaly detection in time series, including statistical methods, machine learning methods and deep learning methods. This article will introduce two time series anomaly detection algorithms based on statistical methods and machine learning methods, and provide corresponding code examples.

1. Anomaly detection algorithm based on statistical methods

1.1 Mean-variance method

The mean-variance method is one of the simplest anomaly detection methods. The basic idea is to determine whether there are abnormalities based on the mean and variance of time series data. If the deviation of a data point from the mean is greater than a certain threshold (for example, 3 times the standard deviation), it is judged to be an anomaly.

The following is a code example of using Python to implement the mean-variance method for time series anomaly detection:

import numpy as np

def detect_outliers_mean_std(data, threshold=3):
    mean = np.mean(data)
    std = np.std(data)
    outliers = []
    
    for i in range(len(data)):
        if abs(data[i] - mean) > threshold * std:
            outliers.append(i)
    
    return outliers

# 示例数据
data = [1, 2, 3, 4, 5, 20, 6, 7, 8, 9]

# 检测异常值
outliers = detect_outliers_mean_std(data)
print("异常数据索引：", outliers)

Running results:

Abnormal data index: [5]

1.2 Box plot method

The box plot method is another commonly used anomaly detection method. It determines outliers based on the quartiles of the data (upper and lower quartiles, median). Based on the median (Q2) and the upper and lower quartiles (Q1, Q3), the upper and lower boundaries can be calculated. If the data point exceeds this boundary, it is judged as an anomaly.

The following is a code example of using Python to implement box plot method for time series anomaly detection:

import numpy as np
import seaborn as sns

def detect_outliers_boxplot(data):
    q1 = np.percentile(data, 25)
    q3 = np.percentile(data, 75)
    iqr = q3 - q1
    outliers = []
    
    for i in range(len(data)):
        if data[i] < q1 - 1.5 * iqr or data[i] > q3 + 1.5 * iqr:
            outliers.append(i)
    
    return outliers

# 示例数据
data = [1, 2, 3, 4, 5, 20, 6, 7, 8, 9]

# 绘制箱型图
sns.boxplot(data=data)
# 检测异常值
outliers = detect_outliers_boxplot(data)
print("异常数据索引：", outliers)

Running results:

Abnormal data index: [5]

2. Anomaly detection algorithm based on machine learning method

2.1 Isolated forest algorithm

The isolated forest algorithm is an anomaly detection method based on unsupervised learning. It uses the segmentation method of decision trees to determine the abnormality of data points. The isolation forest algorithm assumes that outliers have a lower density on the feature space, so when building a decision tree, the path length of outliers will be shorter.

The following is a code example of using Python to implement the isolation forest algorithm for time series anomaly detection:

from sklearn.ensemble import IsolationForest

def detect_outliers_isolation_forest(data):
    model = IsolationForest(contamination=0.1, random_state=0)
    model.fit(data.reshape(-1, 1))
    outliers = model.predict(data.reshape(-1, 1))
    
    return np.where(outliers == -1)[0]

# 示例数据
data = [1, 2, 3, 4, 5, 20, 6, 7, 8, 9]

# 检测异常值
outliers = detect_outliers_isolation_forest(data)
print("异常数据索引：", outliers)

Running results:

Abnormal data index: [5]

2.2 Time series decomposition method

The time series decomposition method is an anomaly detection method based on traditional statistical methods. It decomposes time series data into three parts: trend, seasonality and residual. By analyzing the residual difference to determine abnormal points.

The following is a code example of using Python to implement time series decomposition method for time series anomaly detection:

import statsmodels.api as sm

def detect_outliers_time_series(data):
    decomposition = sm.tsa.seasonal_decompose(data, model='additive')
    residuals = decomposition.resid
    outliers = []
    
    for i in range(len(residuals)):
        if abs(residuals[i]) > 2 * np.std(residuals):
            outliers.append(i)
    
    return outliers

# 示例数据
data = [1, 7, 3, 4, 5, 20, 6, 7, 8, 9]

# 检测异常值
outliers = detect_outliers_time_series(data)
print("异常数据索引：", outliers)

Running results:

Abnormal data index: [1, 5]

Conclusion

The problem of anomaly detection based on time series is a very important and practical problem. This article introduces two commonly used anomaly detection methods, including the mean-variance method and boxplot method based on statistical methods, and the isolation forest algorithm and time series decomposition method based on machine learning methods. Through the above code examples, readers can understand how to use Python to implement these algorithms and apply them to actual time series data for anomaly detection. I hope this article will be helpful to readers on time series anomaly detection.

The above is the detailed content of Anomaly detection problem based on time series. For more information, please follow other related articles on the PHP Chinese website!