From Data to Deployment

DataWhisper: Mastering DL Project Lifecycle

Author: Abdellah Hallou (LinkedIn, Twitter)

Welcome to the Deep Learning Project Starter Guide! This tutorial serves as a comprehensive resource for anyone looking to dive into the exciting world of deep learning. Whether you're a beginner or an experienced developer, this guide will take you through the process of building a deep learning project from start to finish.

Table of contents

• What you'll learn
• Who should follow this tutorial
• Need help or have questions?
• Let's get started!
• Imports and loading the dataset
• Dataset Structure
• Exploratory Data Analysis (EDA)
• Preprocess the data
• Build the model
• Evaluate accuracy
• Save and Export the Model
• Make predictions
• Deployment
  • Create a new flutter project
  • Configuring the Camera
  • Creating the Camera Screen
  • Integrating Image Upload
  • Object Recognition with TensorFlow Lite
  • Running the Model on Images
  • Displaying Results in a Dialog
  • Building the User Interface

What you'll learn

In this tutorial, you will learn the essential steps involved in creating and deploying a deep-learning model in a mobile app. We will cover the following topics:

1. Preparing the data: We'll explore various methods for data preprocessing to ensure a robust and reliable dataset for training.

2. Model creation: You'll discover how to design and build your CNN model.

3. Training the model: We'll delve into the process of training your deep learning model using TensorFlow.

4. Deployment in a mobile app: Once your model is trained, we'll guide you through the steps to integrate it into a mobile app using TensorFlow Lite. You'll understand how to make predictions on the go!

Who should follow this tutorial

This tutorial is suitable for beginners and intermediate developers with a basic understanding of deep learning concepts and Python programming. Whether you're a data scientist, machine learning enthusiast, or mobile app developer, this guide will equip you with the necessary knowledge to kick-start your deep learning project.

Need help or have questions?

If you encounter any issues, have questions, or need further clarification while following this tutorial, don't hesitate to create a GitHub issue in this repository From-Data-to-Deployment. I'll be more than happy to assist you and provide the necessary guidance.

To create an issue, click on the "Issues" tab at the top of this repository's page and click the "New issue" button. Please provide as much context and detail as possible about the problem you're facing or the question you have. This will help me understand your concern better and provide you with a prompt and accurate response.

Your feedback is valuable and can help improve this tutorial for other users as well. So, don't hesitate to reach out if you need any assistance. Let's learn and grow together!

Let's get started!

To start, ensure you have the required dependencies and libraries installed. The tutorial is divided into easy-to-follow sections, each covering a specific aspect of the deep learning project workflow. Feel free to jump to the sections that interest you the most or follow along from beginning to end.

Are you ready?

Imports and loading the dataset

Let's start the necessary imports for our code. We will use the Fashion Mnist dataset in this tutorial.

# Import the necessary libraries
from __future__ import print_function
import keras
from google.colab import drive
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.wrappers.scikit_learn import KerasClassifier
from keras import backend as K
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
from keras.utils.vis_utils import plot_model
import matplotlib.pyplot as plt

Dataset Structure

In any deep learning project, understanding the data is crucial. Before diving into model creation and training, let's start by loading the data and gaining insights into its structure, variables, and overall characteristics.

# Load the Fashion MNIST dataset
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

Exploratory Data Analysis (EDA)

Now that the data is loaded, let's perform some exploratory data analysis to gain a better understanding of its characteristics.

print("Shape of the training data : ",x_train.shape)
print("Shape of the testing data : ",x_test.shape)

Shape of the training data :  (60000, 28, 28)
Shape of the testing data :  (10000, 28, 28)

The Fashion MNIST dataset contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels), as seen here:

60,000 images are used to train the network and 10,000 images to evaluate how accurately the network learned to classify images.

# Printing unique values in training data
unique_labels = np.unique(y_train, axis=0)
print("Unique labels in training data:", unique_labels)

Unique labels in training data: [0 1 2 3 4 5 6 7 8 9]

The labels are an array of integers, ranging from 0 to 9. These correspond to the class of clothing the image represents:
| Label | RClass |
| - |-|
| 0 | T-shirt/top|
| 1 | Trouser|
| 2 |Pullover|
| 3 |Dress|
| 4 |Coat|
| 5 |Sandal|
| 6 |Shirt|
| 7 |Sneaker |
| 8 |Bag|
| 9 | Ankle boot |

Since the class names are not included with the dataset, store them here to use later when plotting the images:

# Numeric labels
numeric_labels = np.sort(np.unique(y_train, axis=0))
# String labels
string_labels = np.array(['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat','Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot'])
# Mapping numeric labels to string labels
numeric_to_string = dict(zip(numeric_labels, string_labels))
print("Numeric to String Label Mapping:")
print(numeric_to_string)

Numeric to String Label Mapping:
{0: 'T-shirt/top', 1: 'Trouser', 2: 'Pullover', 3: 'Dress', 4: 'Coat', 5: 'Sandal', 6: 'Shirt', 7: 'Sneaker', 8: 'Bag', 9: 'Ankle boot'}

Preprocess the data

The data must be preprocessed before training the network.
We start by defining the number of classes in our dataset (which is 10 in this case) and the dimensions of the input images (28x28 pixels).

# Import the necessary libraries
from __future__ import print_function
import keras
from google.colab import drive
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.wrappers.scikit_learn import KerasClassifier
from keras import backend as K
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
from keras.utils.vis_utils import plot_model
import matplotlib.pyplot as plt

This part is responsible for reshaping the input image data to match the expected format for the neural network model. The format depends on the backend being used (e.g., TensorFlow or Theano). In this snippet, we check the image data format using K.image_data_format() and apply the appropriate reshaping based on the result.

# Load the Fashion MNIST dataset
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

The pixel values of the images in the data fall within the range of 0 to 255.
Scale these values to a range of 0 to 1 before feeding them to the CNN model.

print("Shape of the training data : ",x_train.shape)
print("Shape of the testing data : ",x_test.shape)

Convert the class labels (represented as integers) to a binary class matrix format, which is required for multi-class classification problems.

Shape of the training data :  (60000, 28, 28)
Shape of the testing data :  (10000, 28, 28)

Build the model

In this step, we define and build a convolutional neural network (CNN) model for image classification. The model architecture consists of multiple layers such as convolutional, pooling, dropout, and dense layers. The build_model function takes the number of classes, training and testing data as input and returns the training history and the built model.

# Printing unique values in training data
unique_labels = np.unique(y_train, axis=0)
print("Unique labels in training data:", unique_labels)

Unique labels in training data: [0 1 2 3 4 5 6 7 8 9]

# Numeric labels
numeric_labels = np.sort(np.unique(y_train, axis=0))
# String labels
string_labels = np.array(['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat','Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot'])
# Mapping numeric labels to string labels
numeric_to_string = dict(zip(numeric_labels, string_labels))
print("Numeric to String Label Mapping:")
print(numeric_to_string)

Evaluate accuracy

To assess the performance of the trained model, we evaluate it on the test data. The evaluate method is used to calculate the test loss and accuracy. These metrics are then printed to the console.

Numeric to String Label Mapping:
{0: 'T-shirt/top', 1: 'Trouser', 2: 'Pullover', 3: 'Dress', 4: 'Coat', 5: 'Sandal', 6: 'Shirt', 7: 'Sneaker', 8: 'Bag', 9: 'Ankle boot'}

num_classes = 10
# input image dimensions
img_rows, img_cols = 28, 28

if K.image_data_format() == 'channels_first':
    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

Save and Export the Model

After training the model, we save it in the Hierarchical Data Format (HDF5) file format using the save method. The model is then exported to the Google Drive by calling the move_to_drive function. Additionally, the model is converted to the TensorFlow Lite format using the h52tflite function, and the resulting TFLite model is also saved in the Google Drive. The paths of the saved model and TFLite model are returned.

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

Make predictions

To visualize the model's predictions, we select a random set of test images. The model predicts the class labels for these images using the predict method. The predicted labels are then compared with the ground truth labels to display the images along with their corresponding predicted labels using matplotlib.

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

for more information about the model, check these resources:

1. https://www.tensorflow.org/tutorials/keras/classification
2. https://github.com/cmasch/zalando-fashion-mnist/tree/master

Deployment

Create a new Flutter project

Before creating new Flutter project, make sure that the Flutter SDK and other Flutter app development-related requirements are properly installed: https://docs.flutter.dev/get-started/install/windows

After the project has been set up, we will implement the UI to allow users to take pictures or upload images from the gallery and perform object recognition using the exported TensorFlow Lite model.
First, we need to install these packages:

1. camera: 0.10.4
2. image_picker:
3. tflite: ^1.1.2

To do so copy the following code snippet and paste it into the pubspec.yaml file of the project:

# Import the necessary libraries
from __future__ import print_function
import keras
from google.colab import drive
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.wrappers.scikit_learn import KerasClassifier
from keras import backend as K
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
from keras.utils.vis_utils import plot_model
import matplotlib.pyplot as plt

Import the necessary packages in the main.dart file of the project

# Load the Fashion MNIST dataset
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

Configuring the Camera

To enable camera functionality, we'll utilize the camera package. First, import the necessary packages and instantiate the camera controller. Use the availableCameras() function to get a list of available cameras. In this tutorial, we'll use the first camera in the list.

print("Shape of the training data : ",x_train.shape)
print("Shape of the testing data : ",x_test.shape)

Creating the Camera Screen

Create a new StatefulWidget called CameraScreen that will handle the camera preview and image capture functionality. In the initState() method, initialize the camera controller and set the resolution preset. Additionally, implement the _takePicture() method, which captures an image using the camera controller.

Shape of the training data :  (60000, 28, 28)
Shape of the testing data :  (10000, 28, 28)

Integrating Image Upload

To allow users to upload images from the gallery, import the image_picker package. Implement the _pickImage() method, which utilizes the ImagePicker class to select an image from the gallery. Once an image is selected, it can be processed using the _processImage() method.

# Printing unique values in training data
unique_labels = np.unique(y_train, axis=0)
print("Unique labels in training data:", unique_labels)

Object Recognition with TensorFlow Lite

To perform object recognition, we'll use the TensorFlow Lite. Begin by importing the tflite package. In the _initTensorFlow() method, load the TensorFlow Lite model and labels from the assets. You can specify the model and label file paths and adjust settings like the number of threads and GPU delegate usage.

# Import the necessary libraries
from __future__ import print_function
import keras
from google.colab import drive
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.wrappers.scikit_learn import KerasClassifier
from keras import backend as K
from sklearn.model_selection import GridSearchCV
import tensorflow as tf
from keras.utils.vis_utils import plot_model
import matplotlib.pyplot as plt

Running the Model on Images

Implement the _objectRecognition() method, which takes an image file path as input and runs the TensorFlow Lite model on the image. The method returns the label of the recognized object.

# Load the Fashion MNIST dataset
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

Displaying Results in a Dialog

When an image is processed, display the result in a dialog box using the showDialog() method. Customize the dialog to show the recognized object label and provide an option to cancel.

print("Shape of the training data : ",x_train.shape)
print("Shape of the testing data : ",x_test.shape)

Building the User Interface

Shape of the training data :  (60000, 28, 28)
Shape of the testing data :  (10000, 28, 28)

The above is the detailed content of From Data to Deployment. For more information, please follow other related articles on the PHP Chinese website!