Documentation

From pypicloud

pypicloud was the closest thing to velodex in Python: a Pyramid application offering private hosting on S3/GCS/Azure/local storage with a fallback = cache mode that downloaded misses from PyPI, stored them, and served them. Its repository was archived on August 27, 2023 ("Pypicloud has transitioned to maintenance mode"), with the last release in December 2022. It runs today only under Python 3.10 with SQLAlchemy pinned below 2.

Comparison against velodex

Overlap

  • Read-through cache-on-miss. pypicloud's fallback = cache is velodex's default cached-index behavior: fetch a miss, store it, serve it.
  • Private hosting of your own packages, private names taking precedence over public ones.
  • Token or user-authenticated uploads.

Extra: what pypicloud does that velodex does not

  • Cloud storage backends. pypicloud stores artifacts on S3, GCS, Azure Blob, or local disk. velodex stores on local disk only.
  • Pluggable cache and access backends. pypicloud keeps its package index in SQLAlchemy, Redis, or DynamoDB, and drives access through config, SQL, or LDAP user/group systems. velodex embeds its metadata store (redb, nothing to provision) and authenticates uploads with one token per index.
  • Horizontal scale-out. Several stateless pypicloud web servers can share one storage backend and cache DB. velodex is one process per data directory.

Missing: what velodex adds

  • It is maintained. pypicloud is archived and pinned to a pre-2.0 SQLAlchemy stack.
  • A streaming cold path. pypicloud buffers a missed wheel fully into a TemporaryFile, writes it to storage and a cache row, and only then serves it, so the client waits for the whole download plus the disk write plus the DB commit. velodex streams the bytes to the client and into the store at once.
  • Concurrency correctness. A cold burst of clients asking pypicloud for the same wheel each download it and race to insert the same primary key into single-writer SQLite; the losers surface as HTTP 500. velodex single-flights the fetch, so all waiters tail one download.
  • Content-addressed dedup and PEP 658 metadata, neither of which pypicloud offers (it stores files by name/version/filename and serves no .metadata sibling).

Performance vs velodex

The benchmark suite runs both from their published packages. Cold and warm installs through uv:

uv: install the top 51 PyPI packages (ratios vs direct)
velodex pypicloud
cold cache 3.8 s (1.14x) 10.0 s (2.99x)
warm cache 2.7 s (0.83x) 4.4 s (1.36x)
server CPU 1.0 s (1.00x) 3.3 s (3.16x)
server peak memory 537 MB (1.00x) 402 MB (0.75x)

The throughput workload includes the cold burst that pypicloud answers with HTTP 500: four clients ask for one large wheel the instant it lands.

moving one large wheel (torch): cold under contention, hot at speed (ratios vs direct)
velodex pypicloud
cold cache: 4 clients, one wheel 794 ms (0.26x) error (n/a)
hot cache: single download 6,166 MB/s (52.97x) error (n/a)
hot cache: 8 parallel downloads 9,527 MB/s (81.56x) error (n/a)
server CPU 1.1 s (1.00x) 655 ms (0.61x)
server peak memory 54 MB (1.00x) 287 MB (5.28x)

How to migrate

Feature-wise this is the most direct migration: velodex's read-through cached index is pypicloud's fallback = cache made the default. Your cached-index state refills on first use; only hosted uploads need to move. Map the config across:

pypicloudvelodex
ppc-make-config + pserve config.inia TOML file + velodex serve
pypi.fallback = cachethe default cached-index behavior
pypi.fallback = redirect / nonenot offered; misses serve through the cache or 404 on hosted-only indexes
storage = s3 / gcs / azurelocal data_dir only
db = sqlalchemy / redis / dynamo cacheembedded (redb), nothing to provision
access backends (config / SQL / LDAP)one upload_token per hosted index
/simple/ and /pypi/ routes/{route}/simple/

Gotchas

  • No object-storage backend. If your deployment depended on S3 durability, put data_dir on a durable volume and back it up (plain files; rsync works), or wait for a storage backend seam.
  • No per-user permissions. velodex authenticates uploads with a token per index and leaves reads open; see the devpi page for the same caveat.
  • One process per data directory. Multiple stateless web servers sharing one cache was a valid pypicloud shape; it is not a velodex shape. Run one velodex per site.
On this page