Package Manager Development: Registry Design, Cache, Layout, and Linking, Publishing and Authentication¶

Part 2 of 5 of the Package Manager Development reference (plugin package-manager-development), covering §5–§7. Sibling skills: package-manager-versioning-and-resolution (§0–§4), package-manager-supply-chain-and-workspaces (§8–§10), package-manager-ux-ecosystems-and-governance (§11–§14), package-manager-reference (§15–§20). Section numbers are shared across the set; a reference written as §N → skill points into that sibling skill.

Currency: Verified August 2026. See §17 → package-manager-reference for the currency snapshot and what goes stale first.

How to read this. Reference, not tutorial. Sections are independent. Three markers: - [DURABLE] — computer science, or a lesson every ecosystem has independently learned. Does not expire. - [ECOSYSTEM] — specific to npm, PyPI, crates.io, Go, etc. Verify against that ecosystem's current docs. - [CONTESTED] — the ecosystems have made genuinely different, defensible choices, or the practitioners disagree. Both cases given.

⚠️ GOTCHA boxes mark the design mistakes that produce unreproducible builds, wedged caches, exponential resolutions, or supply-chain compromise.

The framing that organizes everything below: a package manager is a distributed systems problem wearing a CLI. It is a cache-coherency problem, a constraint-solving problem, a trust-and-identity problem, and a compatibility-contract problem, all of which happen to be invoked by typing install. Every serious difficulty in this document is one of those four, and treating it as "just downloading files" is the root of most bad designs.

§5. Registry Design¶

5.1 The API surface¶

GET  /packages/{name}                  → all versions + metadata (the resolver's hot path)
GET  /packages/{name}/{version}        → one version's metadata
GET  /packages/{name}/{version}/dl     → the artifact (usually a CDN redirect)
POST /packages                         → publish (authenticated)
DELETE / POST /yank                    → yank/unyank (NOT delete — see 5.2)
GET  /search, /index, /changes         → discovery and incremental sync

Design notes that matter at scale: - Separate metadata from artifacts. Metadata is small, hot, highly cacheable, and resolver-critical. Artifacts are large, cold, and belong on a CDN. Conflating them makes resolution slow (§3.6 → package-manager-versioning-and-resolution). - Support conditional requests and incremental sync (ETag, If-None-Match, a change feed). Mirrors and CI caches depend on it. - Paginate and bound everything. A package with 10,000 versions must not return a 10 MB JSON blob on every resolution step. - Publish upload timestamps (Python's PEP 700 does) — this is what makes cooldowns possible (§8.4 → package-manager-supply-chain-and-workspaces).

5.2 Immutability — the single most important registry policy¶

[DURABLE] A published (name, version) must never change its bytes. Consequences of getting this wrong are severe and every ecosystem has learned it, usually the hard way: - Caches everywhere become incoherent. - Lockfile integrity hashes break for everyone. - The left-pad incident (2016) — an unpublished package broke thousands of builds — established that unpublishing must be tightly restricted.

The correct primitive is a yank (or quarantine), not a delete: - Yanked versions remain downloadable so existing lockfiles keep working. - They are excluded from new resolutions. - crates.io, PyPI (quarantine), and npm's deprecate/unpublish windows all converge on this.

⚠️ And immutability has a security cost you must design for: Go's proxy.golang.org and sum.golang.org provide extremely strong reproducibility — but that same immutability means a malicious module version, once published and cached, persists and continues to be served even after the source repository is removed. Reproducibility and takedown are in direct tension. Have a documented answer for it.

5.3 Mutable-release poisoning — a live design problem¶

Even with immutable versions, most registries historically allowed adding new files to an existing release (e.g. a new wheel for a new Python version, months later). That's a poisoning vector: a stolen token lets an attacker add a malicious artifact to a long-trusted release.

PyPI closed this in July 2026: new files are rejected on releases older than 14 days. The reasoning is instructive — the change was proposed during PEP 740 discussions in January 2024, stalled, and was revived by the March 2026 LiteLLM and Telnyx compromises. Before shipping, PyPI measured the impact: of the top 15,000 projects, only 56 had uploaded a Python 3.14-compatible wheel more than 14 days after the release first appeared. Seth Larson's stated rationale is worth internalizing: it prevents releases from entering "an indeterminate and confusing state of both compromised and not compromised, where only a subset of files could be poisoned."

[DURABLE] That's the template for registry policy changes: identify the vector, measure the legitimate usage you'd break, publish the numbers, then change the default.

5.4 Namespacing and name allocation¶

Model	Example	Trade-off
Flat, first-come	npm (unscoped), PyPI, crates.io	Simple; land-grab, typosquatting, permanent name exhaustion
Scoped/namespaced	`@org/pkg` (npm), NuGet prefixes	Ties identity to an owner; reduces squatting
Domain-derived	Go (`github.com/user/mod`), Maven (reverse-DNS groupId)	Namespace is inherited from an existing trust root. Elegant; couples you to the host
Content-addressed	Nix, Guix	Names are labels, identity is a hash

[DURABLE] Go's and Maven's approach — deriving the namespace from a domain or repo you already control — eliminates an entire class of problems (squatting, name disputes, ownership transfer ambiguity) at the cost of coupling package identity to a hosting provider. If you're designing new, seriously consider it.

Whatever you choose, you need policy for: name similarity (§8.3 → package-manager-supply-chain-and-workspaces), abandoned packages, ownership transfer, trademark disputes, and reuse of deleted names (never reuse a name — that's the "repojacking" attack).

§6. Cache, Layout, and Linking¶

6.1 The content-addressed store¶

[DURABLE] The right shape for a package cache is a global, content-addressed store.

~/.cache/pm/
  index/         metadata, keyed by (name, version), immutable → cacheable forever
  files/         file blobs keyed by content hash (sha256)
  packages/      package trees, assembled from file blobs
  tmp/           staging — assemble here, then atomically rename into place

Properties you get for free: deduplication across projects, integrity verification by construction, safe concurrent access (a hash-named file either exists and is correct, or doesn't exist), and trivial garbage collection.

Write atomically: download to tmp/, verify the hash, then rename() into place. A partially-written cache entry that looks complete is the classic "wedged cache" bug that users fix by deleting the whole cache and cursing your name.

pnpm's contribution was demonstrating how much this matters: a single content-addressed store hard-linked into each project's node_modules means one copy of a given package version on the whole machine, saving developers with many projects tens of gigabytes.

6.2 The layout problem — four answers¶

This is the JavaScript ecosystem's defining argument, and it generalizes.

Layout	Mechanism	Ecosystem	Trade-off
Nested	Each package gets its own `node_modules`	npm v2	Correct; enormous duplication; Windows path-length failures
Flat / hoisted	Everything flattened to the top, conflicts nested	npm v3+, Yarn Classic, Bun (default)	Compact, compatible; allows phantom dependencies (§6.3)
Isolated / symlinked	Content-addressed store + hardlinks + a symlink tree mirroring the true graph	pnpm, Bun `--linker isolated`	Strict — packages can only see what they declared; occasional postinstall scripts assume a hoisted shape and break
No node_modules	A `.pnp.cjs` map from import to zip archive	Yarn Berry PnP	Fastest, strictest, "zero installs"; requires runtime cooperation and breaks tools that stat the filesystem

⚠️ GOTCHA — phantom dependencies are a correctness bug, not a style issue. Under a hoisted layout, require('lodash') works even if you never declared lodash, because something else hoisted it to the top. Your code then breaks when that transitive dependency changes — a failure with no visible cause in your own manifest. Strict layouts exist entirely to make this impossible, and that, not disk space, is their real argument.

[DURABLE] The generalizable lesson: you're choosing between the real dependency graph and a flattened approximation that the runtime finds easier. Flattening is faster and more compatible; it is also lying to the program about what it can see. Ecosystems whose runtime resolution is explicit (Python's sys.path per-venv, Cargo's compiler-passed --extern, Go's import paths) never had this problem, because the layout was never the lookup mechanism.

6.3 Multiple versions coexisting¶

If your runtime can load two versions of the same package simultaneously (npm, Cargo), the resolver can escape most conflicts (§3.5 → package-manager-versioning-and-resolution) — at the cost of duplicated code, larger artifacts, and singleton bugs. If it can't (Python, Java on one classpath), a conflict is fatal and must be reported, which makes resolution failures far more common and your error messages far more important.

Cargo's rule is a good middle: semver-compatible versions are unified into one; semver- incompatible versions coexist. This maximizes deduplication while keeping the escape hatch, and the "expected foo::Type, found foo::Type" error is the price.

6.4 Runtime resolution is part of your design¶

The package manager's job doesn't end at install; the runtime has to find the code. - Node: directory walk up from the importer, looking for node_modules — the reason layout is resolution. - Python: sys.path, one environment per virtualenv — flat, one version of anything. - Cargo/Rust: the compiler is passed explicit --extern paths — no filesystem search at all, which is why Rust can have two versions of a crate without ambiguity. - Go: import path includes the major version (/v2) — semantic import versioning, so two majors are literally different packages. - JVM: classpath order; first match wins. The source of endless "jar hell."

[DURABLE] If you're designing a new ecosystem, make runtime resolution explicit and independent of directory layout. It removes phantom dependencies, path-length limits, and hoisting entirely.

§7. Publishing and Authentication¶

7.1 The evolution, and where it landed¶

Gen 1  username + password                  → phishable, reused, no scoping
Gen 2  long-lived API token                 → phishable, exfiltratable, "harvest now use later"
Gen 3  token + 2FA on the account           → helps login, does NOT help a stolen CI token
Gen 4  TRUSTED PUBLISHING (OIDC)            → no long-lived credential exists at all  ★
       + provenance attestations            → and the artifact proves where it came from

[DURABLE] Trusted publishing is the current best practice and the direction every major registry has moved. The mechanism: 1. The registry pre-registers a trust relationship: "GitHub Actions workflow .github/workflows/release.yml on repo org/repo, environment release, may publish package X." 2. At publish time, CI requests a short-lived OIDC token from its provider. 3. The registry validates the token's claims (repo, workflow, ref/environment) against the registration and issues a short-lived, minimally-scoped upload credential. 4. No long-lived secret is ever stored anywhere.

PyPI shipped this in April 2023; npm added it in response to Shai-Hulud; crates.io has it GA. PEP 807 proposes standardizing the mechanism so any Python index can implement it, encapsulating PyPI's existing implementation.

Its explicit security properties, as PEP 807 states them: credentials are short-lived and minimally scoped, limiting blast radius; automatic expiry means attackers cannot mount "harvest now, use later" campaigns; and the upload is conceptually linked to the CI identity authorized to perform it.

⚠️ GOTCHA — trusted publishing is only as good as its trust configuration. Trust by full repository, not by org or wildcard. Trust by branch or environment, not by any ref. A trusted-publishing config that accepts any workflow on any branch is worse than a token, because it looks secure. The 2026 attacks specifically exploited this class of misconfiguration.

7.2 Provenance and attestations¶

The vocabulary, precisely — people conflate these constantly: - Provenance = a factual claim about how an artifact was built (source repo, commit, builder, parameters). On its own it has no integrity properties whatsoever. - Attestation = provenance wrapped in a signed envelope, so the claim has a cryptographically verifiable author. - The dominant envelope is DSSE (Dead Simple Signing Envelope); the dominant payload format is an in-toto statement with a SLSA provenance predicate. "Attestation" in 2026 almost always means DSSE + in-toto. - SLSA describes build integrity levels; Sigstore is the signing mechanism that makes the claims trustworthy rather than self-asserted text.

Sigstore's keyless model is why this became practical: instead of maintainers managing long-lived signing keys (which historically nobody did well — see the PGP-on-PyPI experience), you generate an ephemeral keypair, exchange a CI OIDC token with the Fulcio CA for a short-lived X.509 certificate binding that key to the CI identity, sign, publish the signature to the Rekor transparency log, and throw the key away. No key management, and anyone can verify both the signature and the identity that made it.

Registry state as of 2026: - npm — Sigstore-backed SLSA provenance, GA since 2023, generated by the npm CLI itself. - PyPI — PEP 740 digital attestations, Final in 2024, generated automatically by pypa/gh-action-pypi-publish v1.11.0+ when using trusted publishing. Over 132,000 PyPI packages carried attestations as of March 2026. - crates.io — trusted publishing is GA, but Sigstore-based signing/provenance (RFC 3403) remains at the proposal stage; integrity comes from elsewhere. - Go — arguably the strongest integrity story of any ecosystem via sum.golang.org, but no build provenance; achieving higher SLSA levels goes through the build platform, not the registry. - Maven Central — PGP signatures traditionally; Sigstore adoption is emerging and Java provenance is the least mature of the major ecosystems. - GitHub Artifact Attestations provide a language-agnostic path (SLSA Build L2 by default, L3 with reusable workflows), which is how ecosystems without registry support get provenance at all.

[DURABLE, and the sentence to remember] Provenance does not prove a package is safe. It proves it came from the build pipeline its maintainers claim. That is precisely enough to detect the dominant 2025–26 attack pattern — a malicious version published from a stolen token outside the project's normal CI — and precisely not enough to detect a maintainer who was themselves compromised (§8.1 → package-manager-supply-chain-and-workspaces).

Plugin: package-manager-development · View SKILL.md on GitHub