Whether you’re a data scientist, software developer, or engineer, you probably encounter Python on a daily basis. Maybe you already know the basics – or maybe you’re fairly proficient at this point. What happens after you learn Python and build a working project? You probably want to publicly share, deploy, or distribute your code for others to install and use. So how do you do this?
PyPI to the rescue! First of all…
What is PyPI?
PyPI (Python Package Index) is an open source repository for all Python projects offered by developers around the world. The main advantage of using PyPI to distribute your work is the ease with which others can install and use your code on their local devices. Distributing a package using PyPI makes your package installable using pip, a command we use to install Python packages on local devices in virtual environments.
Unfortunately, the resources for distributing your work via PyPI are limited and often incomplete. In this article, I’ll simplify the process of distributing a package to PyPI in five simple steps using configuration tools.
5 Easy Steps to Package and Publish Your Python Code to PyPI
- Prepare your code files.
- Prepare your supporting files.
- Build your package locally.
- Upload your package to TestPyPI.
- Publish your work on PyPI.
Step 1: Prepare your code files
To pack your work, you need, well…work. Once you’ve completed your code, modularized it, and tested it, you’ll need to put it into a specific hierarchy before you start packaging and publishing it. Here’s a simple project hierarchy you can start with:
packageName
├── LICENSE
├── projectName
│ ├── __init__.py
│ ├── module_1.py
│ ├── module_1.py
│ └── module_2.py
├── README
├── pyproject.toml
└── setup file
In some packages, the packageName
and the projectName
are the same, but it is not necessary. The packageName
is used in the pip
installation command, so it will be pip
install packageName
while the project name is the one used in the import
command after installing the package. Once the installation is complete, you type import projectName
.
If your project requires greater depth in the hierarchy, be sure to include a__init__.py
file in each hierarchy to make it importable. Basically, if your directory contains an __init__.py
file, you can then import the contents of this file. For example, consider the following directory:
mydir
├── dir
│ ├── __init__.py
│ └── module_1.py
└── some other files
Assuming that mydir
is on your path (the one you choose when installing Python) you will be able to import the code module_1.py
in your code files simply like:
import dir.module_1
#Or
from dir import module_1
When you run Python, if the __init__.py
was not in the directory, the interpreter will no longer look for modules in this directory. This will lead to an import error if you try to import these modules. The __init__.py
file, in most cases, is an empty file. Sometimes it may include simpler names for submodules.
Step 2: Prepare your support files
Now that your files are clean and sorted, we can add the support files. You will need four or five support files to complete your package files.
The installation file
This file includes metadata about the project, including its author, repository, a description of the project, the license it is released under, and more. There are two types of installation files: static and dynamic.
- Static (
setup.cfg
): This means that the file is the same each time you install the package. It has an easy to read and understand format. - Dynamic (
setup.py
): Some elements of the file are dynamic or determined at installation time.
The official Python packaging website strongly suggests using static configuration files and only using dynamic when absolutely necessary. For this reason I will focus on the static configuration file. Here is a model of setup.cfg
file you can use.
[metadata]
name = NAME_OF_YOUR_PROJECT
version = 0.0.1
author = YOUR_NAME
author_email = YOUR_EMAIL
description = WHAT IS THE REASON YOU BUILD THIS PROJECT AND WHAT IT DOES
long_description = file: README.md
long_description_content_type = text/markdown
url = GITHUB REPOSITORY LINK
classifiers =
Programming Language :: Python :: 3
License :: OSI Approved :: Apache Software License
Operating System :: OS Independent
[options]
packages = find:
python_requires = >=3.7
include_package_data = True
The [options]
sections deal with dependencies. Line Package = find:
works automatically to detect your package dependencies.
The pyproject.toml
This is a simple, short, and often fixed file that includes Python’s tools for creating a PyPI package. It basically explains to PyPI that we plan to use configuration tools and wheels to distribute and build our package.
[build-system]
requires = [
“setuptools>=42”,
“wheel”
]
build-backend = “setuptools.build_meta”
A license file
There are many different open source licenses. Which one you want depends on your goal and the nature of your project; you always find help if you have trouble deciding which license to use.
Once you have chosen your license, you must add it to the setup
file in a format accepted by PyPI. In the example above, I used the Apache license. You can find all supported license formats here.
A README file
The README file often includes detailed information about the project, its installation, and perhaps a usage example. In PyPI packages, README files often take one of four forms:
README
README.txtREADME.rst
README.md
The files are either plain text or a restructured text or a Markdown file. Whichever file you decide to use, it will be used as the project description on the package page on PyPI.
MANIFESTO.in
It’s a optional file, and you should only include it if you have unencoded files in your package. If so, you should write them down in this file so that the package installer knows where to find them.
Step 3: Build your package locally
You are almost done. Before uploading your package to PyPI, you need to build it locally and make sure there are no missing files. You also want to make sure there are no errors in your code or in the supporting files. So, from your package directory, run the command line.
If you don’t have the wheel
tool, you will need to install it and have the latest version of build
tool.
pip install wheel
py -m pip install --upgrade build
Now that you have the tool to create the package, you can start creating your project.
py -m build
If all your files are in order, this command should produce many command lines and complete without error.
Step 4: upload your package to TestPyPI
Just because your package was successfully built locally doesn’t mean everything will be fine when you try to pip install
this. So this is just a testing and debugging step; you can skip it if you want. Uploading your package to testPyPI will allow you pip install
it’s just for testing.
So go ahead and sign up for both APIPy and testPyPI. I must point out that these two are completely independent and do not share a database. Once registered, you will have a username and password; be sure to memorize them as you will use them to download your package.
Now install Twine which is a tool to help you create the package.
pip install twine
To upload your project to testPyPI, enter the following command:
py -m twine upload --repository testpypi dist/*
Which should give something similar to:
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: [your username]
Enter your password:
Uploading yourpkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl
100%|█████████████████████| 4.65k/4.65k [00:01<00:00, 2.88kB/s]
Uploading yourpkg_YOUR_USERNAME_HERE-0.0.1.tar.gz
100%|█████████████████████| 4.25k/4.25k [00:01<00:00, 3.05kB/s]
You can then install your package in a virtual environment and test to make sure it works correctly. If so, we can move it to the final stage.
Step 5: Distribute your work on PyPI
Once you’ve made sure your package works on testPyPI, you can go ahead and upload it to PyPI. If this is your first time downloading this package, you can use the following command to download it.
py -m twine upload --repository PyPI dist/*
But if you have already released a package and just need to upload a new version of it, you should use this command.
py -m twine upload --skip-existing dist/*
There you go, your package is downloaded and ready to use.