Publishing Python Packages to PyPI Using Build and Twine

The Problem: Ensuring Secure and Reproducible Package Distribution

Many developers mistakenly use setup.py to both define metadata and handle the upload process. This approach is deprecated and risky, as it mixes the build environment with the distribution environment. The modern standard separates the creation of the distribution files from the act of uploading them, ensuring that what you test locally is exactly what users install via pip.

The takeaway: Use pyproject.toml for configuration, the build module to create standardized artifacts, and twine to upload those artifacts using API tokens.

Prerequisites

• A PyPI account (or a TestPyPI account for initial trials).
• An API Token generated from your PyPI account settings (required for secure authentication).
• Python 3.7+ installed on your local machine.
• A project directory containing your source code and a pyproject.toml file.

Step 1: Define Package Metadata

Avoid using setup.py for metadata. Instead, use pyproject.toml, which is the current PEP 517/518 standard. This file tells the build system exactly what is required to package your code.

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "your-package-name"
version = "0.1.0"
description = "A short description of your library"
dependencies = [
    "requests>=2.25.1",
]

Step 2: Build Distribution Artifacts

You must generate two types of files: the Source Distribution (sdist), which contains the raw source code, and the Wheel (.whl), a built distribution that installs faster.

Run the following command in your project root using a terminal with your virtual environment activated:

python -m build

Expected check: Verify that a dist/ directory now exists containing both a .tar.gz (sdist) and a .whl file for your version.

Risk: If your package includes C extensions, the wheel generated will be specific to your current OS and architecture. For cross-platform binary support, you would need a tool like cibuildwheel.

Step 3: Validate the Distribution

Before uploading, check that your long description (usually pulled from a README) will render correctly on the PyPI website. A formatting error in your Markdown or ReST can result in a plain-text page on PyPI.

# Run this from the project root
twine check dist/*

Step 4: Secure Upload to PyPI

Use twine to upload the files located in the dist/ directory. Do not use your account password; use the API token.

twine upload dist/*

When prompted for a username, enter __token__. For the password, paste your API token (including the pypi- prefix). This prevents your actual account password from being stored in shell histories or configuration files.

Verification and Diagnostics

To confirm the release was successful, attempt to install the specific version in a fresh, isolated virtual environment:

python -m venv test_env
source test_env/bin/activate
pip install your-package-name==0.1.0

Symptom	Cause	Resolution
HTTP 400 Error	Version already exists on PyPI	Increment version in pyproject.toml and rebuild.
Missing files in install	Files excluded by default	Create a MANIFEST.in to explicitly include non-Python files.
Authentication Failure	Incorrect username	Ensure username is exactly __token__.

Rollback and Corrections

PyPI does not allow you to overwrite an existing version. If you discover a critical bug or a security leak (e.g., accidentally uploading a .env file), you cannot "update" that version number.

1. Yank the release: Use the PyPI web interface to "yank" the version. This prevents new users from installing it but allows existing dependencies to remain functional.
2. Correct the source: Fix the code or remove the sensitive files.
3. Re-release: Increment the version number (e.g., 0.1.0 to 0.1.1) and repeat the build and upload process.