Python Deep Learning Guide

Deep learning is a prominent topic in the field of artificial intelligence and has been attracting attention for quite some time. It has attracted much attention because of breakthroughs beyond human capabilities in fields such as computer vision (Computer Vision) and gaming (Alpha GO). Since the last survey, interest in deep learning has increased significantly.

The image below is what Google Trends shows us:

This article is a good non-technical one if you are interested in this topic introduce. If you are interested in learning about the latest trends in deep learning, this article is a comprehensive summary.

Here, our goal is to provide a learning path for newbies and people who want to explore deep learning further. So, are you ready to embark on a journey to conquer deep learning? Let's hit the road.

Step 0: Prepare for the expedition

It is recommended that you should understand the basic knowledge of machine learning before entering the field of in-depth learning. "Learning Paths to Machine Learning" is a complete resource to get you started in the field.

If you want a shorter version, take a look below:

Basic Mathematics, Resource 1: "Mathematics | Khan Academy" (especially Calculus, Probability, and Linear Algebra )

Python basics, resources: "Introduction to Computer Science", EDX course

Basics of Statistics, resources: "Introduction to Statistics", Udacity's course

Machine Learning Basics, Resources: "Introduction to Machine Learning," Udacity's course

Time: 2-6 months recommended

Step 1: Set up your machine

Before proceeding to the next step , please make sure you have supporting hardware. The general recommendation should be at least:

A good enough GPU (4+ GB), preferably Nvidia

A sufficient CPU (such as Intel Core i3 is OK, but Intel Pentium is not) )

4GB memory or depending on the dataset.

If you’re still not sure, take a look at this hardware guide

PS: If you’re a hardcore gamer (and not just a Candy Crush player of course!), you probably already have what you need hardware.

If you don’t have the required hardware, you can purchase or rent an Amazon Web Service (AWS). Here is a good guide to using AWS for deep learning.

Note: There is no need to install any deep learning learning library at this stage. This is done in step 3.

Step 2: Try it

Now that you have enough preliminary knowledge, you can learn more about deep learning.

According to personal preference, you can follow:

Blog: (Resource 1: "Basics of Deep Learning" Resource 2: "Hacker's Guide to Neural Networks")

Video: "Deep Learning Simplified"

Textbook: "Neural Networks and Deep Learning"

In addition to these prerequisites, you should also know popular deep learning libraries and the languages ​​that run them. Here is a (non-comprehensive) list (see the wiki page for a more comprehensive list):

Caffe

DeepLearning4j

Tensorflow

Theano

Torch

Some other famous libraries include:

Mocha

neon

H2O

MXNet

Keras

Lasagne

Nolearn

Here is a list of deep learning libraries classified according to language

It is recommended to read Stanford’s cs231n course Lecture 12: Popular Overview of Deep Learning Libraries

Time: 1-3 weeks recommended

Step 3: Choose your own path of adventure

Now comes the fun part! Deep learning has been applied in various fields and has brought state-of-the-art results. In order to enter this field, you, the reader, need to choose a path that suits you. This should be a practical experience so that you gain a suitable foundation on top of your current understanding.

Note: Each path includes an introductory blog, a practical project, the deep learning program library and auxiliary courses required for the project. Start by understanding the introduction, then install the required libraries and start working on your project. If you encounter any difficulty with this, take advantage of the relevant courses for support.

Computer Vision of Deep Learning

Introduction: "DL for Computer Vision" Computer Vision Blog

Project: "FacialKeypoint Detection" Facial Keypoint Detection Tutorial

Required library: Nolearn

Related courses: "cs231n: Convolutional Neural Network Visual Recognition"

Natural Language Processing of Deep Learning

Introduction to Introduction: "DeepLearning, NLP, and Representations" ("Deep Learning, Natural Language Processing and Representation" blog)

Project: "Deep Learning Chatbot":

"Part One"

"Second Part of "

Required libraries: tensorflow

Related courses: "cs224d: Application of Deep Learning in Natural Language Processing"

Deep Learning for Speech/Audio Recognition

Introduction: "Deep Talk: Lessons Learned from Deep Learning" news and corresponding video.

Project: "Generating music with Magenta (tensorflow)"

Required libraries: Magenta,

Related courses: "Deep Learning (Spring 2016), cilvr Lab @ New York

Deep Learning Reinforcement Learning

Introduction and project: "Deep Reinforcement Learning: Table Tennis Game from Pixels"

Required libraries: No requirements. But you need openAI gym to test your model

Related courses: "cs294: Deep Reinforcement Learning"

Time: 1-2 months recommended

Step 4: In-depth. Deep Learning is now (almost) worthy of a spot in the Deep Learning Hall of Fame! The road ahead is long and deep (pun intended), and is largely unexplored right now. As you get familiar with and use this new skill. Here are some tips to improve your skills.

Choose a different path to deep learning not mentioned above! (Examples: deep learning for trading, deep learning for optimizing energy efficiency)

Use your newly learned skills to create something (remember, with great power comes great responsibility, )

Test your deep learning skills (such as Kaggle)

Participate in deep learning communities (such as Google Group)

Follow the latest research results and researchers (such as "RE.WORK DL Summit". ”)

Time: Suggestion - no upper limit!

Specifically, let’s talk about using python to develop or learn machine learning, an important module of deep learning! It should be said to be an indispensable module!

1. Preliminary preparation, you need to know more or less python scientific operations, data sorting and image result modules. These three are indispensable: numpy, pandas, matplotlib.

2. Advanced preparation, in In terms of machine learning, scikit learn is a module that brings together many machine learning methods. It supports a variety of machine learning methods.

3. It is also advanced in neural networks. Tensorflow, which has made great contributions, was developed by Google and has hired many Theano developers. I think there is no one more powerful in the development of neural networks than Tensorflow, and if you think about Google, the speed of updates will definitely be the same. It won't be slow. Look forward to it.

4. If you want to build a neural network quickly and easily, Keras is a very good module. Its underlying layer is Tensorflow and Theano, so it can be used on Windows, MacOS, and Linux.
5. There are indeed many other modules that can be used, but in my personal machine learning career, these are the main ones I use.
In order to facilitate everyone's learning and promote the power of Chinese people in the field of machine learning, I have made my own set of scikit learn, Theano, Keras, and tensorflow Chinese teaching videos, and an introduction series to machine learning. The response in China has not been great, but many people have watched it on YouTube. I hope everyone will support the power of the Chinese people.

Summary of Python machine learning library and deep learning library

1. Scikit-learn (highly recommended)

Scikit-learn is based on Scipy is a Python module built for machine learning. It features a variety of classification, regression and clustering algorithms including support vector machines, logistic regression, naive Bayes classifiers, random forests, Gradient Boosting, and clustering algorithms. and DBSCAN. And also designed Python numerical and scientific libraries Numpy and Scipy

2. Keras (deep learning)

Keras is a deep learning framework based on Theano , its design refers to Torch, written in Python language, and is a highly modular neural network library that supports GPU and CPU.

3. Lasagne (Deep Learning)

is not only a delicious Italian dish, but also a deep learning library with similar functions to Keras, but its design is somewhat different from them.

4.Pylearn2

Pylearn is a Theano-based library program that simplifies machine learning research. It encapsulates many commonly used models and training algorithms in deep learning and artificial intelligence research into a single experimental package, such as stochastic gradient descent.

5.NuPIC

NuPIC is a machine intelligence platform using HTM learning algorithm as a tool. HTM is an accurate calculation method for the cortex. The core of HTM is a time-based continuous learning algorithm and a spatiotemporal pattern of storage and recall. NuPIC is suitable for a wide variety of problems, especially anomaly detection and prediction from streaming data sources.

6. Nilearn

Nilearn is a Python module that can quickly statistically learn neuroimaging data. It uses the scikit-learn toolbox in the Python language and some applications for predictive modeling, classification, decoding, and connectivity analysis to perform multivariate statistics.

7.PyBrain

Pybrain is the abbreviation of reinforcement learning, artificial intelligence, and neural network library based on Python language. Its goal is to provide flexible, easy-to-use, and powerful machine learning algorithms and to test your algorithms in a variety of predefined environments.

8.Pattern

Pattern is a network mining module in the Python language. It provides tools for data mining, natural language processing, network analysis and machine learning. It supports vector space models, clustering, support vector machines and perceptrons and uses the KNN classification method for classification.

9.Fuel

Fuel provides data for your machine learning model. It has an interface for sharing data sets such as MNIST, CIFAR-10 (image data set), Google's One Billion Words (text). You use it to replace your own data in a variety of ways.

10.Bob

Bob is a free signal processing and machine learning tool. Its toolbox is co-written in Python and C++. It is designed to be more efficient and reduce development time. It is composed of a large number of software packages for processing imaging tools, audio and video processing, machine learning and pattern recognition. of.

11.Skdata

Skdata is a library program for machine learning and statistical data sets. This module provides standard Python usage for toy problems, popular computer vision and natural language datasets.

12.MILK

MILK is a machine learning toolkit in Python language. It mainly uses supervised classification methods among the many available classifications such as SVMS, K-NN, random forests, and decision trees. It also performs feature selection. These classifiers are combined in many ways to form different classification systems such as unsupervised learning, affinity gold propagation, and K-means clustering supported by MILK.

13.IEPY

IEPY is an open source information extraction tool focusing on relationship extraction. It is mainly aimed at users who need to extract information from large data sets and scientists who want to try new algorithms.

14.Quepy

Quepy is a Python framework that queries in a database query language by changing natural language questions. It can be simply defined as different types of problems in natural language and database queries. So, you can build your own system that feeds natural language into your database without coding.

Now Quepy provides support for Sparql and MQL query languages. And there are plans to extend it to other database query languages.

15.Hebel

Hebel is a library program for deep learning of neural networks in the Python language. It uses PyCUDA for GPU and CUDA accelerate. It is the most important type of tool for neural network models and can provide activation functions for a number of different activity functions, such as dynamics, Nesterov dynamics, loss of signal and stopping methods.

16.mlxtend

It is a library program composed of useful tools and extensions for daily data science tasks.

17.nolearn

This package contains a large number of utility modules that can help you complete machine learning tasks. A large number of these modules work with scikit-learn, and others are often more useful.

18.Ramp

Ramp is a library program that develops solutions for accelerating prototyping in machine learning in the Python language. It is a lightweight pandas-based pluggable framework for machine learning. It provides a simple declarative syntax exploration function for existing machine learning and statistical tools in the Python language (such as scikit-learn, rpy2, etc.) Implement algorithms and transformations quickly and efficiently.

19.Feature Forge

This series of tools creates and tests machine learning functions through an API compatible with scikit-learn.

This library provides a set of tools that will make you useful in many machine learning programs. When you use scikit-learn this tool, you will feel that you have been greatly helped. (Although this only works if you have different algorithms.)

20.REP

REP is about conducting in a harmonious, reproducible way An environment provided by the data movement driver.

It has a unified classifier wrapper to provide a variety of operations, such as TMVA, Sklearn, XGBoost, uBoost, etc. And it can train classifiers in a parallel manner on a population. It also provides an interactive plot.

21.Python Learning Machine Sample

A collection of simple software built with Amazon’s machine learning.

22.Python-ELM

This is an implementation of an extreme learning machine based on scikit-learn in the Python language.

23.gensim

topic model python implementation

Scalable statistical semantics

Analyze plain-text documents for semantic structure

Retrieve semantically similar documents