Private Python Packages: Complete Setup Checklist

Private Python packages let you share proprietary code within your team while keeping it secure. They help with access control, dependency management, and ensure consistent workflows. Here's a quick overview of how to set them up:

• Organize your package: Create a clear folder structure with essential files like pyproject.toml and __init__.py.
• Host on a private repository: Use platforms like GitHub Packages or GitLab for secure hosting.
• Build & publish: Use tools like Poetry to package and publish your code.
• Install securely: Use pip with repository URLs and access tokens for installation.
• Automate with CI/CD: Streamline updates with GitHub Actions or similar tools.

Best Practices:

• Use short-lived tokens for security.
• Stick to clear versioning (e.g., 1.0.0 for major updates).
• Automate dependency checks and security scans.

Top Platforms:

Platform	Key Features	Cost
GitHub Packages	Seamless GitHub integration	$4/user
GitLab Registry	Built-in CI/CD support	$19/user
Envelope	Python-specific hosting	$5/month

Private packages are essential for secure, efficient, and scalable Python development. Ready to set yours up? Let’s dive in!

Private Python Package Artifact Registry with Google Cloud

Checklist for Setting Up Private Python Packages

Follow this checklist to set up secure and team-friendly private Python packages.

1. Organize Your Package Structure

Start with a clear and well-organized structure. Include key files like pyproject.toml for configuration and __init__.py to define the package. Here's an example:

my-package/
├── src/my_package/
│   └── __init__.py
└── pyproject.toml

Your pyproject.toml can look like this:

[tool.poetry]
name = "my-package"
version = "0.1.0"

2. Set Up a Private Repository

Use a platform like GitHub Packages to host your private Python packages. Start by generating a Personal Access Token for authentication. Then, configure your ~/.pypirc file:

[distutils]
index-servers = github

[github]
repository = https://github.com/api/v4/projects/<project-id>/packages/pypi
username = __token__
password = <your-personal-access-token>

3. Build and Publish Your Package

Leverage Poetry to build and publish your package:

poetry build
poetry publish --repository github

4. Install Private Packages

Use pip to install your private package. Include the repository URL and token:

pip install --index-url https://__token__:<token>@github.com/api/v4/projects/<project-id>/packages/pypi/simple package_name

5. Automate with GitHub Actions

Streamline your workflow by automating package publishing. Create a GitHub Actions workflow file (.github/workflows/publish.yml):

name: Publish Package
on:
  release:
    types: [created]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
      - name: Build and publish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          pip install poetry
          poetry build
          poetry publish

Once your private package setup is complete, focus on maintaining it effectively for long-term use.

Best Practices for Managing Private Python Packages

Secure Repository and Access Tokens

Use short-lived or package-specific tokens instead of permanent ones to enhance security. Store these tokens safely using environment variables:

export GITHUB_TOKEN="your-token"
pip install --index-url https://${GITHUB_TOKEN}@github.com/<repo-path>/simple package_name

For organizations, consider tools that manage token lifecycles, such as setting expiration dates and automating token rotation. These steps help protect your private packages while enabling smooth collaboration across teams.

Version Control and Dependency Management

Stick to clear versioning practices (e.g., 1.0.0 for major updates) to ensure predictable package updates. Tools like Poetry make managing dependencies straightforward:

[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.28.0"
pandas = ">=1.4.0,<2.0.0"

If Poetry isn't an option, pip-tools provides another effective solution:

pip-compile pyproject.toml  # Create locked dependency files
pip-sync requirements.txt  # Synchronize your environment

Keep Packages Updated and Monitored

Set up automated security scans for your packages to catch vulnerabilities early:

- name: Security Scan
  run: |
    pip install safety
    safety check

Track package usage through server logs and establish a clear policy for phasing out older versions:

• Latest version: Fully supported
• Previous major version: Security fixes for 6 months
• Older versions: Marked as deprecated

Regularly check for outdated packages using pip list --outdated and maintain a changelog to document major changes. This keeps your team informed about updates and their impact.

With these practices in place, you're better prepared to choose the right platform for managing private Python packages effectively.

How to Choose a Platform for Private Python Packages

Picking the right platform for hosting your private Python packages involves weighing factors like security, integration options, and cost. The platform you choose will shape how secure, scalable, and efficient your package management process becomes.

Compare Hosting Platforms

Here’s a breakdown of some popular private Python package hosting platforms and what they offer:

Platform	Key Features	Security	Monthly Cost	Storage/Bandwidth
GitHub Packages	Seamless GitHub integration, GPG signatures	2FA, Access controls	$4/user	Free (public), Paid (private)
GitLab Package Registry	Built-in CI/CD, supports multiple package types	Deploy tokens, Access controls	$19/user	Free tier
Envelope	Python-specific hosting, aligns with Python repository standards	Time-limited keys, Detailed permissions	$5 (Basic)	1GB/10GB

GitHub Packages stands out for its strong integration with GitHub repositories, making it a go-to option for teams already using GitHub. According to the 2023 State of the Octoverse report, 45% of organizations rely on it for private package hosting.

GitLab Package Registry shines in environments that need robust CI/CD workflows. For example, Liip uses its built-in CI/CD tools and deploy tokens for secure package access, simplifying their deployment process.

Envelope is tailored for Python developers, offering features like unlimited access keys and time-sensitive permissions. Its $5/month Basic plan includes 1GB of storage and 10GB of bandwidth, making it a practical choice for smaller teams.

When deciding on a platform, focus on these key areas:

• Integration: Check if the platform works smoothly with your tools and CI/CD setup.
• Security: Features like 2FA, access controls, and package verification are essential.
• Scalability: Ensure the platform can handle your growing storage and bandwidth needs.

For teams that need full control, self-hosted options like PyPI Server offer flexibility but come with added maintenance responsibilities. Managed solutions often save time and money when you factor in infrastructure and upkeep costs.

Once you’ve chosen the right platform, the next step is to focus on managing and maintaining your private Python packages effectively.

Conclusion: Managing Private Python Packages Effectively

Handling private Python packages well means focusing on security, efficiency, and automation. A solid approach can simplify workflows while keeping sensitive code safe.

Security should always come first. Using packages from reliable sources and managing dependencies carefully helps avoid vulnerabilities. Beyond installations, securing your infrastructure is just as important to protect private packages.

A strong private package system is built around three key areas:

• Infrastructure Security: Use TLS, store credentials in secure environments, and rely on tools like pip-compile to keep dependencies safe and reliable.

• Automation and Monitoring: Implement CI/CD pipelines to minimize manual work, and monitor regularly to catch dependency conflicts or security risks early.

• Version Control and Dependencies: Tools like poetry or setuptools help maintain consistent dependencies, reducing errors as your package ecosystem grows.

Treat private packages like production assets. Set up proper access controls, keep documentation up to date, and create clear processes for updates and handling vulnerabilities.

If you're new to private packages, begin with basic security measures and gradually add automation. Focus on reproducible builds and clear dependency management. Make regular security checks and updates a standard part of your workflow.

Private package management isn’t a one-time task - it’s an ongoing effort. Stay updated on security practices and use automation to keep your Python package ecosystem secure and efficient.

FAQs

Can PyPI packages be private?

No, PyPI (Python Package Index) is a public repository, meaning all packages are visible to everyone. If you need private package hosting, consider platforms like GitHub Packages or GitLab. These platforms allow secure hosting using personal access tokens. Here's an example of how to install a package from a private GitHub repository:

pip install --index-url https://<token>@github.com/org/repo/simple package_name

How to create a private PyPI server?

If your team needs full control over package hosting, setting up a private PyPI server is a great option. Here’s how you can do it:

• Set up the server: Install a tool like PyPiServer and create a directory for your packages.
• Enable HTTPS: Secure your server with an SSL certificate, either self-signed or issued by a trusted authority.
• Configure the web server: Use Apache or Nginx to serve your packages efficiently.
• Control access: Add authentication and configure pip to connect to your private server.

To use your private server, update your requirements.txt file or use the --index-url option with pip to point to your repository. While platforms like GitHub offer managed hosting solutions, a private PyPI server gives you more control but requires additional maintenance.

For more tips on securing access tokens and managing repository permissions, check out the Best Practices section mentioned earlier.