A brief analysis of active learning of cell image data

Set priorities and weights for data through the impact of cell image labels on model performance.

One of the major obstacles for many machine learning tasks is the lack of labeled data. Labeling data can take a long time and be expensive, so many times it is unreasonable to try to use machine learning methods to solve the problem.

In order to solve this problem, a field called active learning emerged in the field of machine learning. Active learning is a method in machine learning that provides a framework for prioritizing unlabeled data samples based on the labeled data that the model has already seen. If you want to

Segmentation and classification technologies for cell imaging are a rapidly developing field of research. Just like in other machine learning fields, data annotation is very expensive, and the quality requirements for data annotation are also very high. To address this problem, this article introduces an active learning end-to-end workflow for red blood cell and white blood cell image classification tasks.

Our goal is to combine biology and active learning and help others use active learning methods to solve similar and more complex tasks in biology.

This article mainly consists of three parts:

• Cell image preprocessing - here we will introduce how to preprocess unsegmented blood cell images.
• Use CellProfiler to extract cell features - shows how to extract morphological features from biological cell photo images to be used as features for machine learning models.
• Use active learning - shows a comparative experiment simulating the use of active learning and not using active learning.

Cell image preprocessing

We will use the blood cell image dataset licensed under MIT (GitHub and Kaggle). Each image is labeled according to red blood cell (RBC) and white blood cell (WBC) classification. There are additional tags for these four types of leukocytes (eosinophils, lymphocytes, monocytes, and neutrophils), but these were not used in this study.

Here is an example of a full-size original image from the dataset:

Create sample DF

The original dataset contains an export .py script that parses the XML annotations into a CSV table containing the filename, cell type label, and bounding box for each cell.

The original script did not include the cell_id column, but we wanted to classify individual cells, so we modified the code slightly to add that column and added a filename column that included image_id and cell_id:

import os, sys, randomimport xml.etree.ElementTree as ETfrom glob import globimport pandas as pdfrom shutil import copyfileannotations = glob('BCCD_Dataset/BCCD/Annotations/*.xml')df = []for file in annotations:#filename = file.split('/')[-1].split('.')[0] + '.jpg'#filename = str(cnt) + '.jpg'filename = file.split('\')[-1]filename =filename.split('.')[0] + '.jpg'row = []parsedXML = ET.parse(file)cell_id = 0for node in parsedXML.getroot().iter('object'):blood_cells = node.find('name').textxmin = int(node.find('bndbox/xmin').text)xmax = int(node.find('bndbox/xmax').text)ymin = int(node.find('bndbox/ymin').text)ymax = int(node.find('bndbox/ymax').text)row = [filename, cell_id, blood_cells, xmin, xmax, ymin, ymax]df.append(row)cell_id += 1data = pd.DataFrame(df, columns=['filename', 'cell_id', 'cell_type', 'xmin', 'xmax', 'ymin', 'ymax'])data['image_id'] = data['filename'].apply(lambda x: int(x[-7:-4]))data[['filename', 'image_id', 'cell_id', 'cell_type', 'xmin', 'xmax', 'ymin', 'ymax']].to_csv('bccd.csv', index=False)

Crop

To be able to process the data, the first step is to crop the full-size image based on the bounding box coordinates. This produces many images of cells of varying sizes:

The cropped code is as follows:

import osimport pandas as pdfrom PIL import Imagedef crop_cell(row):"""crop_cell(row)given a pd.Series row of the dataframe, load row['filename'] with PIL,crop it to the box row['xmin'], row['xmax'], row['ymin'], row['ymax']save the cropped image,return cropped filename"""input_dir = 'BCCDJPEGImages'output_dir = 'BCCDcropped'# open imageim = Image.open(f"{input_dir}{row['filename']}")# size of the image in pixelswidth, height = im.size# setting the points for cropped imageleft = row['xmin']bottom = row['ymax']right = row['xmax']top = row['ymin']# cropped imageim1 = im.crop((left, top, right, bottom))cropped_fname = f"BloodImage_{row['image_id']:03d}_{row['cell_id']:02d}.jpg"# shows the image in image viewer# im1.show()# save imagetry:im1.save(f"{output_dir}{cropped_fname}")except:return 'error while saving image'return cropped_fnameif __name__ == "__main__":# load labels csv into Pandas DataFramefilepath = "BCCDdataset2-masterlabels.csv"df = pd.read_csv(filepath)# iterate through cells, crop each cell, and save cropped cell to filedataset_df['cell_filename'] = dataset_df.apply(crop_cell, axis=1)

The above are all the preprocessing operations we have done. Now, we continue to use CellProfiler to extract features.

Extract cell features using CellProfiler

CellProfiler is a free and open source image analysis software that can automatically make quantitative measurements from large-scale cell images. CellProfiler also contains a GUI interface that allows us to perform visual operations

First download CellProfiler. If CellProfiler cannot be opened, you may need to install the Visual C release package. Please refer to the official website for specific installation methods.

Open the software to load the image. If you want to build a pipeline, you can find the list of available functions provided by CellProfiler on its official website. Most functions are divided into three main groups: image processing, target processing and measurement.

Commonly used functions are as follows:

Image processing - Convert to grayscale image:

Object target processing - Identify main objects

测量 - 测量对象强度

CellProfiler可以将输出为CSV文件或者保存指定数据库中。这里我们将输出保存为CSV文件，然后将其加载到Python进行进一步处理。

说明：CellProfiler还可以将你处理图像的流程保存并进行分享。

主动学习

我们现在已经有了训练需要的搜有数据，现在可以开始试验使用主动学习策略是否可以通过更少的数据标记获得更高的准确性。 我们的假设是：使用主动学习可以通过大量减少在细胞分类任务上训练机器学习模型所需的标记数据量来节省宝贵的时间和精力。

主动学习框架

在深入研究实验之前，我们希望对modAL进行快速介绍： modAL是Python的活跃学习框架。 它提供了Sklearn API，因此可以非常容易的将其集成到代码中。 该框架可以轻松地使用不同的主动学习策略。 他们的文档也很清晰，所以建议从它开始你的一个主动学习项目。

主动学习与随机学习

为了验证假设，我们将进行一项实验，将添加新标签数据的随机子抽样策略与主动学习策略进行比较。开始用一些相同的标记样本训练2个Logistic回归估计器。然后将在一个模型中使用随机策略，在第二个模型中使用主动学习策略。

我们首先为实验准备数据，加载由Cell Profiler言创建的特征。 这里过滤了无色血细胞的血小板，只保留红和白细胞（将问题简化，并减少数据量） 。所以现在我们正在尝试解决二进制分类问题 - RBC与WBC。使用Sklearn Label的label encoder进行编码，并拆分数据集进行训练和测试。

# imports for the whole experimentimport numpy as npfrom matplotlib import pyplot as pltfrom modAL import ActiveLearnerimport pandas as pdfrom modAL.uncertainty import uncertainty_samplingfrom sklearn import preprocessingfrom sklearn.metrics import , average_precision_scorefrom sklearn.linear_model import LogisticRegression# upload the cell profiler features for each celldata = pd.read_csv('Zaretski_Image_All.csv')# filter plateletsdata = data[data['cell_type'] != 'Platelets']# define the labeltarget = 'cell_type'label_encoder = preprocessing.LabelEncoder()y = label_encoder.fit_transform(data[target])# take the learning features onlyX = data.iloc[:, 5:]# create training and testing setsX_train, X_test, y_train, y_test = train_test_split(X.to_numpy(), y, test_size=0.33, random_state=42)

下一步就是创建模型

<span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">dummy_learner</span> <span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">=</span> <span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">LogisticRegression</span>()<br><span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">active_learner</span> <span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">=</span> <span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">ActiveLearner</span>(<br><span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">estimator</span><span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">=</span><span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">LogisticRegression</span>(),<br><span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">query_strategy</span><span style="color: rgb(215, 58, 73); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">=</span><span style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">uncertainty_sampling</span>()<br>)

dummy_learner是使用随机策略的模型，而active_learner是使用主动学习策略的模型。为了实例化一个主动学习模型，我们使用modAL包中的ActiveLearner对象。在“estimator”字段中，可以插入任何sklearnAPI兼容的模型。在query_strategy '字段中可以选择特定的主动学习策略。这里使用“uncertainty_sampling()”。这方面更多的信息请查看modAL文档。

将训练数据分成两组。第一个是训练数据，我们知道它的标签，会用它来训练模型。第二个是验证数据，虽然标签也是已知的，但是我们假装不知道它的标签，并通过模型预测的标签和实际标签进行比较来评估模型的性能。然后我们将训练的数据样本数设置成5。

# the training size that we will start withbase_size = 5# the 'base' data that will be the training set for our modelX_train_base_dummy = X_train[:base_size]X_train_base_active = X_train[:base_size]y_train_base_dummy = y_train[:base_size]y_train_base_active = y_train[:base_size]# the 'new' data that will simulate unlabeled data that we pick a sample from and label itX_train_new_dummy = X_train[base_size:]X_train_new_active = X_train[base_size:]y_train_new_dummy = y_train[base_size:]y_train_new_active = y_train[base_size:]

我们训练298个epoch，在每个epoch中，将训练这俩个模型和选择下一个样本，并根据每个模型的策略选择是否将样本加入到我们的“基础”数据中，并在每个epoch中测试其准确性。因为分类是不平衡的，所以使用平均精度评分来衡量模型的性能。

在随机策略中选择下一个样本，只需将下一个样本添加到虚拟数据集的“新”组中，这是因为数据集已经是打乱的的，因此不需要在进行这个操作。对于主动学习，将使用名为“query”的ActiveLearner方法，该方法获取“新”组的未标记数据，并返回他建议添加到训练“基础”组的样本索引。被选择的样本都将从组中删除，因此样本只能被选择一次。

# arrays to accumulate the scores of each simulation along the epochsdummy_scores = []active_scores = []# number of desired epochsrange_epoch = 298# running the experimentfor i in range(range_epoch):# train the models on the 'base' datasetactive_learner.fit(X_train_base_active, y_train_base_active)dummy_learner.fit(X_train_base_dummy, y_train_base_dummy)# evaluate the modelsdummy_pred = dummy_learner.predict(X_test)active_pred = active_learner.predict(X_test)# accumulate the scoresdummy_scores.append(average_precision_score(dummy_pred, y_test))active_scores.append(average_precision_score(active_pred, y_test))# pick the next sample in the random strategy and randomly# add it to the 'base' dataset of the dummy learner and remove it from the 'new' datasetX_train_base_dummy = np.append(X_train_base_dummy, [X_train_new_dummy[0, :]], axis=0)y_train_base_dummy = np.concatenate([y_train_base_dummy, np.array([y_train_new_dummy[0]])], axis=0)X_train_new_dummy = X_train_new_dummy[1:]y_train_new_dummy = y_train_new_dummy[1:]# pick next sample in the active strategyquery_idx, query_sample = active_learner.query(X_train_new_active)# add the index to the 'base' dataset of the active learner and remove it from the 'new' datasetX_train_base_active = np.append(X_train_base_active, X_train_new_active[query_idx], axis=0)y_train_base_active = np.concatenate([y_train_base_active, y_train_new_active[query_idx]], axis=0)X_train_new_active = np.concatenate([X_train_new_active[:query_idx[0]], X_train_new_active[query_idx[0] + 1:]], axis=0)y_train_new_active = np.concatenate([y_train_new_active[:query_idx[0]], y_train_new_active[query_idx[0] + 1:]], axis=0)

结果如下：

plt.plot(list(range(range_epoch)), active_scores, label='Active Learning')plt.plot(list(range(range_epoch)), dummy_scores, label='Dummy')plt.xlabel('number of added samples')plt.ylabel('average precision score')plt.legend(loc='lower right')plt.savefig("models robustness vs dummy.png", bbox_inches='tight')plt.show()

策略之间的差异还是很大的，可以看到主动学习只使用25个样本就可以达到平均精度0.9得分！ 而使用随机的策略则需要175个样本才能达到相同的精度！

In addition, the score of the model with active learning strategy is close to 0.99, while the score of the random model stops at around 0.95! If we use all the data, then their final scores are the same, but the purpose of our research is to train on the premise of a small amount of labeled data, so only 300 random samples in the data set are used.

Summary

This paper demonstrates the benefits of using active learning for cell imaging tasks. Active learning is a set of methods in machine learning that prioritize solutions for unlabeled data examples based on the impact their labels have on model performance. Since labeling data is a task involving many resources (money and time), it is necessary to judge which samples to label which can maximize the performance of the model.

Cell imaging has made tremendous contributions to the fields of biology, medicine and pharmacology. In the past, analyzing cell images required valuable professional human capital, but the emergence of technologies like active learning provides a very good solution for fields such as medicine that require large amounts of human-annotated data sets.

The above is the detailed content of A brief analysis of active learning of cell image data. For more information, please follow other related articles on the PHP Chinese website!