Create your own Python package

Python is a great programming language and so much more. However, one of its weakest points is the packaging. This is a well-known fact in society. Installing, importing, using, and creating packages has improved over the years, but it still doesn't compare to newer languages ​​like Go and Rust, which could learn a lot from the struggles of Python and other more mature languages.

In this tutorial, you'll learn everything you need to build and share your own packages. For general background information on Python packages, read How to use Python packages.

Packaging Project

Packaging a project is a process by which you take a consistent set of Python modules and possibly other files and put them into a structure that can be easily used. You must consider factors such as dependencies on other packages, internal structure (subpackages), version control, target audience, and the form of the package (source code and/or binaries).

Example

Let's start with a simple example. The conman package is a package for managing configuration. It supports multiple file formats as well as distributed configuration using etcd.

The contents of a package are usually stored in a single directory (although it is common to split subpackages into multiple directories), and sometimes (as in this example) in its own git repository.

The root directory contains various configuration files (setup.py is required and the most important one), and the package code itself is usually located in a subdirectory whose name is preferably the name of the package. Test directory. This is what conman looks like:

> tree
.
├── LICENSE
├── MANIFEST.in
├── README.md
├── conman
│   ├── __init__.py
│   ├── __pycache__
│   ├── conman_base.py
│   ├── conman_etcd.py
│   └── conman_file.py
├── requirements.txt
├── setup.cfg
├── setup.py
├── test-requirements.txt
├── tests
│   ├── __pycache__
│   ├── conman_etcd_test.py
│   ├── conman_file_test.py
│   └── etcd_test_util.py
└── tox.ini

​

Let’s take a quick look at the setup.py file. It imports two functions from the setuptools package: setup() and find_packages(). It then calls the setup() function with find_packages() as one of the arguments.

from setuptools import setup, find_packages
setup(name='conman',
      version='0.3',
      url='https://github.com/the-gigi/conman',
      license='MIT',
      author='Gigi Sayfan',
      author_email='the.gigi@gmail.com',
      description='Manage configuration files',
      packages=find_packages(exclude=['tests']),
      long_description=open('README.md').read(),
      zip_safe=False,
      setup_requires=['nose>=1.0'],
      test_suite='nose.collector')

​

This is normal. While the setup.py file is a regular Python file in which you can do whatever you want, its main job is to call setup() with the appropriate arguments. function argument as it will be called in a standard way by various tools when installing the package. I'll go into more detail in the next section.

Configuration file

In addition to setup.py, there are some other optional configuration files that can be displayed here and used for various purposes.

setup.py

setup() The function takes a number of named arguments to control many aspects of package installation and to run various commands. A number of parameters specify metadata used for searching and filtering when uploading packages to the repository.

• name: The name of your package (and how it will be listed on PyPI)
• version: This is critical to maintaining proper dependency management
• url: URL of the package, usually GitHub or readthedocs URL
• packages: List of subpackages that need to be included; find_packages() Provide help here
• setup_requires: Specify dependencies here
• test_suite: Which tool to run during testing

long_description Set here to the contents of the README.md file, which is a best practice for having a single source of truth.

setup.cfg

setup.py The file also provides a command line interface to run various commands. For example, to run a unit test, you would type: python setup.py test

​

running test 

running egg_info
writing conman.egg-info/PKG-INFO
writing top-level names to conman.egg-info/top_level.txt
writing dependency_links to conman.egg-info/dependency_links.txt
reading manifest file 'conman.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'conman.egg-info/SOURCES.txt'
running build_ext
test_add_bad_key (conman_etcd_test.ConManEtcdTest) ... ok
test_add_good_key (conman_etcd_test.ConManEtcdTest) ... ok
test_dictionary_access (conman_etcd_test.ConManEtcdTest) ... ok
test_initialization (conman_etcd_test.ConManEtcdTest) ... ok
test_refresh (conman_etcd_test.ConManEtcdTest) ... ok
test_add_config_file_from_env_var (conman_file_test.ConmanFileTest) ... ok
test_add_config_file_simple_guess_file_type (conman_file_test.ConmanFileTest) ... ok
test_add_config_file_simple_unknown_wrong_file_type (conman_file_test.ConmanFileTest) ... ok
test_add_config_file_simple_with_file_type (conman_file_test.ConmanFileTest) ... ok
test_add_config_file_simple_wrong_file_type (conman_file_test.ConmanFileTest) ... ok
test_add_config_file_with_base_dir (conman_file_test.ConmanFileTest) ... ok
test_dictionary_access (conman_file_test.ConmanFileTest) ... ok
test_guess_file_type (conman_file_test.ConmanFileTest) ... ok
test_init_no_files (conman_file_test.ConmanFileTest) ... ok
test_init_some_bad_files (conman_file_test.ConmanFileTest) ... ok
test_init_some_good_files (conman_file_test.ConmanFileTest) ... ok
----------------------------------------------------------------------
Ran 16 tests in 0.160s
OK

setup.cfg is an ini format file that may contain option defaults for commands passed to setup.py. Here, setup.cfg contains some options for nosetests (our test runner):

[nosetests]
verbose=1
nocapture=1

​

MANIFEST.in

This file contains files that are not part of the internal package directory, but that you still want to include. These are usually readme files, license files, etc. An important file is requirements.txt. pip uses this file to install other required packages.

This is conman’s MANIFEST.in file:

include LICENSE
include README.md
include requirements.txt

​

Dependencies

You can specify dependencies in the install_requires section of setup.py and in the requirements.txt file. Pip will automatically install dependencies from install_requires but not from the requirements.txt file. To install these requirements, you must specify it explicitly when running pip: pip install -r requests.txt.

install_requires 选项旨在指定主要版本级别的最低和更抽象的要求。 requirements.txt 文件用于更具体的要求，通常包含固定的次要版本。

这是conman的需求文件。您可以看到所有版本都已固定，这意味着如果其中一个软件包升级并引入破坏 conman 的更改，它可能会受到负面影响。

PyYAML==3.11
python-etcd==0.4.3
urllib3==1.7
pyOpenSSL==0.15.1
psutil==4.0.0
six==1.7.3

 

固定让您可预测且安心。如果许多人在不同时间安装您的软件包，这一点尤其重要。如果没有固定，每个人都会根据安装时间获得不同的依赖版本组合。固定的缺点是，如果您不跟上依赖项的开发，您可能会陷入某些依赖项的旧的、性能不佳甚至易受攻击的版本。

我最初在2014年写了conman，并没有太关注它。现在，在本教程中，我升级了所有内容，并且几乎每个依赖项都进行了一些重大改进。

发行版

您可以创建源发行版或二进制发行版。我将涵盖两者。

源分布

使用以下命令创建源发行版：python setup.py sdist。这是 conman 的输出：

> python setup.py sdist
running sdist
running egg_info
writing conman.egg-info/PKG-INFO
writing top-level names to conman.egg-info/top_level.txt
writing dependency_links to conman.egg-info/dependency_links.txt
reading manifest file 'conman.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'conman.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt
running check
creating conman-0.3
creating conman-0.3/conman
creating conman-0.3/conman.egg-info
making hard links in conman-0.3...
hard linking LICENSE -> conman-0.3
hard linking MANIFEST.in -> conman-0.3
hard linking README.md -> conman-0.3
hard linking requirements.txt -> conman-0.3
hard linking setup.cfg -> conman-0.3
hard linking setup.py -> conman-0.3
hard linking conman/__init__.py -> conman-0.3/conman
hard linking conman/conman_base.py -> conman-0.3/conman
hard linking conman/conman_etcd.py -> conman-0.3/conman
hard linking conman/conman_file.py -> conman-0.3/conman
hard linking conman.egg-info/PKG-INFO -> conman-0.3/conman.egg-info
hard linking conman.egg-info/SOURCES.txt -> conman-0.3/conman.egg-info
hard linking conman.egg-info/dependency_links.txt -> conman-0.3/conman.egg-info
hard linking conman.egg-info/not-zip-safe -> conman-0.3/conman.egg-info
hard linking conman.egg-info/top_level.txt -> conman-0.3/conman.egg-info
copying setup.cfg -> conman-0.3
Writing conman-0.3/setup.cfg
creating dist
Creating tar archive
removing 'conman-0.3' (and everything under it)

 

如您所见，我收到一条关于缺少带有标准前缀之一的 README 文件的警告，因为我喜欢 Markdown，所以我有一个 README.md 。除此之外，还包括所有包源文件和附加文件。然后，在conman.egg-info目录中创建了一堆元数据。最后，创建一个名为 conman-0.3.tar.gz 的压缩 tar 存档，并将其放入 dist 子目录中。

安装这个包需要一个构建步骤（即使它是纯Python）。您可以正常使用 pip 安装它，只需传递包的路径即可。例如：

pip install dist/conman-0.3.tar.gz
Processing ./dist/conman-0.3.tar.gz
Installing collected packages: conman
  Running setup.py install for conman ... done 

Successfully installed conman-0.3

 

Conman 已安装到站点包中，可以像任何其他包一样导入：

import conman
conman.__file__
'/Users/gigi/.virtualenvs/conman/lib/python2.7/site-packages/conman/__init__.pyc'

 

轮子

Wheels 是一种相对较新的方式来打包 Python 代码和可选的 C 扩展。它们取代了鸡蛋的形式。轮子有几种类型：纯Python轮子、平台轮子、通用轮子。纯 Python 轮子是像 conman 这样的包，没有任何 C 扩展代码。

平台轮子确实有 C 扩展代码。通用轮子是纯 Python 轮子，兼容具有相同代码库的 Python 2 和 Python 3（它们甚至不需要 2 到 3）。

如果您有一个纯 Python 包，并且希望您的包同时支持 Python 2 和 Python 3（变得越来越重要），那么您可以构建一个通用构建，而不是为 Python 2 构建一个轮子，为 Python 构建一个轮子3.

Python 3 是当前受到积极支持的 Python 版本，不断更新、改进和社区支持；建议对所有新项目和迁移使用 Python 3。

如果你的包有C扩展代码，你必须为每个平台构建一个平台轮。轮子的巨大好处，特别是对于带有 C 扩展的软件包来说，是不需要在目标机器上提供编译器和支持库。该轮子已经包含一个内置包。所以你知道它不会构建失败，而且安装速度要快得多，因为它实际上只是一个副本。使用 Numpy 和 Pandas 等科学库的人可以真正体会到这一点，因为安装此类软件包过去需要很长时间，并且如果缺少某些库或编译器配置不正确，则可能会失败。

构建纯轮子或平台轮子的命令是：python setup.py bdist_wheel。

Setuptools——提供 setup() 函数的引擎——将自动检测是否需要纯轮子或平台轮子。

running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/conman
copying conman/__init__.py -> build/lib/conman
copying conman/conman_base.py -> build/lib/conman
copying conman/conman_etcd.py -> build/lib/conman
copying conman/conman_file.py -> build/lib/conman
installing to build/bdist.macosx-10.9-x86_64/wheel
running install 

running install_lib
creating build/bdist.macosx-10.9-x86_64
creating build/bdist.macosx-10.9-x86_64/wheel
creating build/bdist.macosx-10.9-x86_64/wheel/conman
copying build/lib/conman/__init__.py -> build/bdist.macosx-10.9-x86_64/wheel/conman
copying build/lib/conman/conman_base.py -> build/bdist.macosx-10.9-x86_64/wheel/conman
copying build/lib/conman/conman_etcd.py -> build/bdist.macosx-10.9-x86_64/wheel/conman
copying build/lib/conman/conman_file.py -> build/bdist.macosx-10.9-x86_64/wheel/conman
running install_egg_info
running egg_info
creating conman.egg-info
writing conman.egg-info/PKG-INFO
writing top-level names to conman.egg-info/top_level.txt
writing dependency_links to conman.egg-info/dependency_links.txt
writing manifest file 'conman.egg-info/SOURCES.txt'
reading manifest file 'conman.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'conman.egg-info/SOURCES.txt'
Copying conman.egg-info to build/bdist.macosx-10.9-x86_64/wheel/conman-0.3-py2.7.egg-info
running install_scripts
creating build/bdist.macosx-10.9-x86_64/wheel/conman-0.3.dist-info/WHEEL

 

查看dist目录，可以看到创建了一个纯Python的轮子：

ls -la dist
dist/
total 32
-rw-r--r--  1 gigi  staff   5.5K Feb 29 07:57 conman-0.3-py2-none-any.whl
-rw-r--r--  1 gigi  staff   4.4K Feb 28 23:33 conman-0.3.tar.gz

 

名称 conman-0.3-py2-none-any.whl 有几个组件：

• 包名称
• 软件包版本
• Python版本
• 平台版本，最后是“whl”扩展。

要构建通用包，您只需添加 --universal，如 python setup.py bdist_wheel --universal。

生成的轮子名为 conman-0.3-py2.py3-none-any.whl。

请注意，如果您创建通用包，您有责任确保您的代码实际上可以在 Python 2 和 Python 3 下运行。

使用 Toml

Toml（Tom's Obvious Minimal Language）是一种易于使用的配置文件格式，用于配置软件应用程序。尽管 setup.py 仍然被广泛使用和支持，PEP518 建议使用 pyproject.toml 文件而不是 setup.py 进行打包和分发。

让我们使用pyproject.toml创建一个名为 apex 的简单 Python 包，用于检查电池状态并在电池充满时显示一条消息。

创建一个名为 apex 的目录结构。在 apex 内，添加一个 pyproject.toml 和一个 README.md 文件。

.apex
├── pyproject.toml
└── README.md

在根 apex 目录中，创建一个名为 apex 的子目录，其中包含一个 __init__.py 文件以使其成为一个包和一个 battery.py 文件。您的项目目录现在应如下所示：

.
├── apex
│   ├── battery.py
│   └── __init__.py
├── pyproject.toml
└── README.md

使用 pip 安装 psutil 包。

pip install psutil

Psutil（进程和系统实用程序）是一个用于 Python 中进程和系统监控的跨平台库。 Psutil 可以检索有关系统上运行的进程的信息，例如状态、CPU 使用情况和内存使用情况。它还可以操纵系统进程。

在battery.py文件中，添加检查电池是否已充满的代码。

import psutil

def check_battery():
    battery = psutil.sensors_battery()
    plugged = battery.power_plugged
    percent = battery.percent

    if percent == 100 and plugged:
        print "Battery is full. Unplug your Charger"
    else:
        print "Battery is not yet full."

if __name__ == "__main__":
    message = check_battery()
    print(message)

pyproject.toml 文件包含包的元数据，包括：

• 构建系统
• 名称
• 版本
• 项目依赖项.

打开pyproject.toml并指定 setuptools 作为构建系统。

[build-system]
requires = ["setuptools","wheel"]
build-backend ="setuptools.build_meta"

指定包的名称、版本号和项目依赖项列表。现在 pyproject.toml 文件如下所示：

[build-system]
requires = ["setuptools","wheel"]
build-backend ="setuptools.build_meta"

[project]
name="apex"
version ="1.0.0"

[project-dependencies]
psutil ="5.9.5"

最后，在README.md文件中添加以下内容。

# apex

**apex** is a Python package that allows you to check the battery status of your laptop and display a message when the battery is full.

## Features

- Retrieve battery status information
- Display a message when the battery is full
- Cross-platform support (Windows, macOS, Linux, etc.)

## Installation

Install **apex** using `pip`:

```shell
pip install apex

使用 build 命令构建包。

 python -m build

您应该看到下面的输出，该输出已被我截断。

creating build/bdist.linux-x86_64/wheel/apex-1.0.0.dist-info/WHEEL
creating '/home/vaati/Desktop/apex/dist/.tmp-d8_lqaaj/apex-1.0.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'apex/__init__.py'
adding 'apex/battery.py'
adding 'apex-1.0.0.dist-info/METADATA'
adding 'apex-1.0.0.dist-info/WHEEL'
adding 'apex-1.0.0.dist-info/top_level.txt'
adding 'apex-1.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built apex-1.0.0.tar.gz and apex-1.0.0-py3-none-any.whl

构建会创建一个 dist 文件夹，其中包含包的分发文件：

• apex-1.0.0.tar.gz：apex 包的源发行版。它包含安装该软件包所需的源代码。
• apex-1.0.0-py3-none-any.whl：apex 包的二进制发行版。您可以使用二进制发行版直接从源代码安装它。

构建过程还会生成一个 apex.egg-info 文件夹，其中包含有关包的元数据。

.
├── apex
│   ├── battery.py
│   └── __init__.py
├── apex.egg-info
│   ├── dependency_links.txt
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   └── top_level.txt
├── dist
│   ├── apex-1.0.0-py3-none-any.whl
│   └── apex-1.0.0.tar.gz
├── pyproject.toml
└── README.md

结论

编写自己的 Python 包需要处理大量工具，指定大量元数据，并仔细考虑您的依赖项和目标受众。但回报是巨大的。

如果您编写有用的代码并正确打包它，人们将能够轻松安装它并从中受益。

The above is the detailed content of Create your own Python package. For more information, please follow other related articles on the PHP Chinese website!