Home  >  Article  >  Backend Development  >  Detailed introduction to Python technology stack and tool organization

Detailed introduction to Python technology stack and tool organization

黄舟
黄舟Original
2017-05-07 11:09:113112browse


Development environment

Editor

  • ##vim / SublimeText2 / PyCharm

    Use Just make it easy, I converted from vim to PyChram. The integrated development environment has much better functions such as auto-completion and single-step debugging, which help improve work efficiency.

Local environment

  • pip/easy_install package management

  • viertualenv + virtualenvwrapper library/version management , Environment isolation

  • ipython/ipdb

Project development

Management tools

  • git

Web

Framework

ORM

  • SQLAlchemy: Standard.

  • pymongo: Access

    mongodb.

  • peewe: A lighter ORM, simple to understand, never used in production environment.

Database

  • Relational databasemysql

No SQL:

Distributed Storage

  • HDFS: hadopp ecology

  • Hive: Analysis log

Message

Queue

  • RabbitMQ: pika operation in python.

Project deployment

Server

  • ##nginx

    , mainly used for load balancing, reverse generation, etc.

  • uWSGI

    , used to deploy Django projects.

  • gunicorn

    a Python WSGI HTTP Server for UNIX, used to run the Flask project

  • Operation and maintenance management

  • saltstack

    : Alias, salt stack. Automated operation and maintenance tools.

  • puppet

    : This product was developed in Ruby and is used on a large scale by Baidu and Xiaomi.

  • fabric

    : Used for automated deployment.

  • Supervisor

    A Process Control System, configures and manages various programs, process monitoring, automatic restart, etc.

  • Three-party library

  • requests

    HTTP for humans, very easy to use, highly recommended

  • beautifulsoup

    Cooperate with urllib2 or requests library for simple crawling and analysis work

  • ##scrapy
  • Very awesome crawling framework, Suitable for large-scale crawling tasks with complex requirements

    Others
Front-end basics

html, css,

    Design Pattern
  • : Although Python is not like the endless design patterns in Java, the basic Design patterns are also used. Combination,

    single case mode, decorator mode, factory mode are commonly used.

  • RESTful Interface.

  • MVC

  • Test: Unit test, performance test.
    Only by comparison can there be differences. Look at other people's codes and learn from them to improve.

Cloud computing

  • Big data: Hadoop ecosystem.

  • Virtualization: Docker, KVM, OpenStack.

  • Public cloud: AWS, Alibaba Cloud, Azure, Kingsoft Cloud.

  • Private cloud: Baidu's private cloud is well built and leads the industry in distributed storage and virtualization.

Development environment

Editor

  • vim / SublimeText2 / PyCharm
    Use Just make it easy, I converted from vim to PyChram. The integrated development environment has much better functions such as auto-completion and single-step debugging, which help improve work efficiency.

Local environment

  • pip/easy_install package management

  • viertualenv + virtualenvwrapper library/version management , Environment isolation

  • ipython/ipdb

Project development

Management tools

  • git

Web Framework

  • Tornado: asynchronous, high performance, latest version 4.0.

  • Flask: Lightweight! It can flexibly combine various components for development (third-party components are abundant), it is simple and efficient, and facilitates rapid development and maintenance.

  • Django: Somewhat heavy, with numerous configurations and conventions, you can quickly develop some "management" backends.
    There are also many Python Web frameworks, and these three are currently the mainstream ones. The Tornado currently used in my work has excellent performance.

ORM

  • SQLAlchemy: Standard.

  • pymongo: access mongodb.

  • peewe: A lighter ORM, simple to understand, never used in production environment.

Database

  • Relational database: mysql

No SQL:

  • redis cache/persistence/special requirements (counting-ranking-timeline, etc.)

  • memcached cluster, mostly used for time-limited cache

  • mongodb

Distributed storage

  • HDFS: hadopp ecology

  • Hive: Analysis log

Message queue

  • RabbitMQ: pika in python operate.

Project deployment

Server

  • ##nginx, mainly used for load balancing, reverse generation, etc.

  • uWSGI, used to deploy Django projects.

  • gunicorn a Python WSGI HTTP Server for UNIX, used to run the Flask project

Operation and maintenance management

  • saltstack: Alias, salt stack. Automated operation and maintenance tools.

  • puppet: This product was developed in Ruby and is used on a large scale by Baidu and Xiaomi.

  • fabric: Used for automated deployment.

  • Supervisor A Process Control System, configures and manages various programs, process monitoring, automatic restart, etc.

Three-party library

  • requests HTTP for humans, very easy to use, highly recommended

  • beautifulsoup Cooperate with urllib2 or requests library for simple crawling and analysis work

  • ##scrapy

    Very awesome crawling framework, Suitable for large-scale crawling tasks with complex requirements

  • Others

Front-end basics

    html, css, javascript , jquery, bootstrap, angularjs, react, vue.js. As a back-end engineer, it is also necessary to understand some basic front-end knowledge. In my current work, I use
  • bootstrap

    +angularjs to develop the backend management system.

  • Software Engineering

    Design patterns: Although Python does not have endless design patterns in Java, basic design patterns are also used. . Combination, singleton mode, decorator mode, factory mode are commonly used.
  • RESTful interface.
  • MVC
  • Testing: unit testing, performance testing.
  • Only by comparison can there be differences. Look at other people's codes and learn from them to improve.


  • Cloud computing

    Big data: Hadoop ecosystem.
  • Virtualization: Docker, KVM, OpenStack.
  • Public cloud: AWS, Alibaba Cloud, Azure, Kingsoft Cloud.
  • Private cloud: Baidu's private cloud is well built and leads the industry in distributed storage and virtualization.

The above is the detailed content of Detailed introduction to Python technology stack and tool organization. For more information, please follow other related articles on the PHP Chinese website!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn