# How to recover when one registry upload fails mid release > Make each publish step ask the registry and skip what it holds, give each registry its own job, push the tag last, and know when to burn a half-published version. Source: https://voxgig.com/howto/recover-from-a-partial-multi-registry-release - Audience: platform-team - Level: advanced - Languages: typescript, javascript - Verified: 2026-10-21 - Published: 2026-10-19 ## Short answer Make each publish step idempotent. Ask the registry whether the version is there, compare its digest with the artifact you built, and skip it when they match. An "already published" refusal means read and compare again, never success. Give each registry its own CI job so a rerun repeats only the failed one, and push the tag last. A wrong artifact that landed cannot be replaced, so burn the version and ship the next patch everywhere. --- ## You will need Node 22 or later, a release that sends one version to more than one registry, and credentials for each. Verified 2026-10-21 against Node 22.22.0. The sample runs against a stand-in for six registries on 127.0.0.1, which answers with the paths, status codes, and refusal texts of the real ones, so it publishes nothing. The upload itself stays with the tool you already run, `npm publish`, `twine upload`, `gem push`, or `cargo publish`; the sample replaces the loop around it. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [All-or-nothing staged bundle, Central Portal style](https://central.sonatype.org/publish/publish-portal-guide/) | The release is Java, or the Java artifacts are the ones that must land together | Only Maven Central offers the staged bundle, so every other registry still needs one of the rows below | Any registry that takes one upload per package and keeps it for good | | [Burn the version and publish a fresh patch](https://pypi.org/help/#file-name-reuse) | An artifact that landed is wrong, or you cannot tell whether it is | A version number consumers see but cannot install, plus a changelog entry explaining the gap | Every landed artifact matches the build, so the version can be finished | | [Per-registry jobs, re-running only the failed ones](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs) | GitHub Actions, or any CI whose rerun repeats only the jobs that failed | A build job that hands its artifacts to the publish jobs, and one runner start per registry | Your CI offers one job, or the registries must publish in a strict order | | [Skip-if-published check in every publish step](https://twine.readthedocs.io/en/stable/) | Any job shape and any CI, and the only guard that still holds when the whole job reruns | One read per package before every upload, and a stopped run while a registry's read API lags a write it already took | The registry has no read API to ask, which is rare | The skip check and the per-registry jobs are not rivals: the jobs limit how much a rerun repeats, and the check makes whatever it repeats harmless. The Central Portal bundle costs you its scope, one registry out of six. Burning the version costs a number consumers see but cannot install, and it is the only choice left once wrong bytes have landed, because no registry here replaces them. ## Ask the registry before you upload Every registry here has a read API that says whether a version exists, and four of the six also say which bytes it holds. `registries.mjs` records the question and the answer for each one: the path to ask, the digest field, the matching local digest, and the refusal sent for a repeat. ```js title="registries.mjs" // alreadyThere is true when the answer to an upload means "this version is held". None // of them says whose bytes are held, so release.mjs reads again and compares digests. export const REGISTRIES = { npm: { kind: 'upload', // GET // is 200 with a dist object, or 404 "version not found". existsPath: (pkg, version) => `/${encodeURIComponent(pkg.name)}/${version}`, read: (status, body) => (status === 200 ? { exists: true, digest: JSON.parse(body).dist.integrity } : { exists: false }), localDigest: sri512, uploadMethod: 'PUT', uploadPath: (pkg) => `/${encodeURIComponent(pkg.name)}`, // A second publish of a version is refused with 403, whatever its bytes. alreadyThere: (status, body) => status === 403 && /previously published/.test(body), }, pypi: { kind: 'upload', // GET /pypi///json is 200 with urls[].digests, or 404. A release // can hold several files, so one that lacks this file has not got it yet. existsPath: (pkg, version) => `/pypi/${pkg.name}/${version}/json`, read: (status, body, pkg) => { if (status !== 200) return { exists: false } const file = JSON.parse(body).urls.find((u) => u.filename === pkg.artifact) return file ? { exists: true, digest: file.digests.sha256 } : { exists: false } }, localDigest: sha256, uploadMethod: 'POST', uploadPath: () => '/legacy/', // Warehouse answers an identical file with 200 and the same file name with other // bytes with 400, putting the message in the status line, where twine reads it. alreadyThere: (status, body, reason) => status === 400 && /already exist/i.test(`${reason} ${body}`), }, ``` The [npm registry API](https://github.com/npm/registry/blob/main/docs/REGISTRY-API.md) answers `GET //` with that version's `dist` object, and `npm publish` sends a sha512 `integrity` field with every tarball, so the comparison is an SRI string. PyPI's [JSON API](https://docs.pypi.org/api/json/) lists each file of a release with its `sha256`. The [RubyGems.org API](https://guides.rubygems.org/rubygems-org-api/) returns one version with its `sha`, and the crates.io [sparse index](https://doc.rust-lang.org/cargo/reference/registry-index.html) returns one JSON line per version with a `cksum` of the `.crate` file. The two tag-driven registries answer yes or no: the [Go module proxy](https://go.dev/ref/mod#goproxy-protocol) serves `@v/.info` once it has seen the tag, and [Packagist](https://packagist.org/apidoc) lists every tagged version in `p2//.json`. Each refusal says the version is held without saying whose bytes it holds. npm answers any repeat with 403, crates.io with a 400 carrying an `errors` array. Warehouse, the code behind PyPI, accepts an identical file with 200 and refuses the same name with other bytes with a 400 whose message rides in the HTTP status line.[^1] twine's `--skip-existing` skips on that 400, so it skips the one upload whose bytes differ. RubyGems.org answers 200 when the same key pushes the same bytes, and 409 otherwise.[^2] ## Skip what landed, stop at what failed, tag last `release.mjs` is the loop a CI job runs. For each upload target it probes, compares, and either skips or uploads. A refusal meaning already there sends it back to read and compare. Any other failure ends the run, a failed read included, since only a 404 or 410 says absent. The tag goes last, because for Go and Packagist the tag is the publication. ```js title="release.mjs" const checkBytes = (there) => { if (there.digest !== spec.localDigest(bytes)) { throw new ReleaseError(`${pkg.registry} holds ${pkg.name} ${plan.version} with different bytes`, { burn: true }) } } const there = await probe(spec, bases[pkg.registry].read, pkg, plan.version, fetch) if (there.exists) { checkBytes(there) log(row(pkg, plan.version, 'already there, skipped')) continue } const url = `${bases[pkg.registry].write}${spec.uploadPath(pkg)}?version=${plan.version}&file=${pkg.artifact}` const res = await fetch(url, { method: spec.uploadMethod, body: bytes }) const body = await res.text() if (spec.alreadyThere(res.status, body, res.statusText)) { // The refusal says the version is held, not whose bytes it holds. Ask again. const again = await probe(spec, bases[pkg.registry].read, pkg, plan.version, fetch) if (!again.exists) { throw new ReleaseError(`${pkg.registry} says ${pkg.name} ${plan.version} is there, but its read API does not show it yet; rerun once it does`) } checkBytes(again) log(row(pkg, plan.version, `already there (${res.status} on upload), same bytes`)) continue } if (!res.ok) throw new ReleaseError(`${pkg.registry}: ${pkg.name} ${plan.version} failed, ${res.status} ${body.split('\n')[0]}`) log(row(pkg, plan.version, 'published')) } if (!tagFirst) await pushTag() ``` Run it twice against the stand-in. The first run is told to fail at crates.io once. ```bash node demo.mjs ``` ```text output run 1: release 1.4.0, crates.io refuses its upload once npm @acme/sdk 1.4.0 published pypi acme-sdk 1.4.0 published rubygems acme_sdk 1.4.0 published stopped: crates: acme-sdk 1.4.0 failed, 503 Service Unavailable serving 1.4.0: npm yes, pypi yes, rubygems yes, crates no, goproxy no, packagist no run 2: the same release again npm @acme/sdk 1.4.0 already there, skipped pypi acme-sdk 1.4.0 already there, skipped rubygems acme_sdk 1.4.0 already there, skipped crates acme-sdk 1.4.0 published tag v1.4.0 pushed: goproxy and packagist now serve 1.4.0 serving 1.4.0: npm yes, pypi yes, rubygems yes, crates yes, goproxy yes, packagist yes upload attempts: npm 1, pypi 1, rubygems 1, crates 2 (one refused) ``` The first run stopped before the tag, so the two tag-driven registries served nothing and no consumer saw a half release. The second run made one read per landed package and uploaded to crates.io alone. The attempt counts on the last line are the property to assert: no registry received a second successful upload. ## Give each registry its own job The skip check makes a rerun harmless, and per-registry jobs make it short. `release.yml` builds once, hands the artifacts to a matrix of publish jobs, and tags in a job that needs all of them. ```yaml title="release.yml" publish: needs: build runs-on: ubuntu-latest permissions: contents: read id-token: write strategy: fail-fast: false matrix: registry: [npm, pypi, rubygems, crates] steps: - uses: actions/checkout@v6 with: persist-credentials: false - uses: actions/download-artifact@v5 with: name: dist path: dist - run: node release/publish.mjs ${{ matrix.registry }} "$(cat VERSION)" tag: needs: publish runs-on: ubuntu-latest permissions: contents: write steps: - uses: actions/checkout@v6 - run: | git tag "v$(cat VERSION)" git push origin "v$(cat VERSION)" ``` [`needs`](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-jobs) holds the tag job back, because a job that fails or is skipped skips every job that needs it. `fail-fast: false` matters because the [default](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax) cancels every in-progress job in the matrix when one fails, which turns one failed upload into several interrupted ones. The [artifact](https://docs.github.com/en/actions/tutorials/store-and-share-data) from `build` is what every publish job uploads, so a rerun publishes the bytes the first run built rather than a rebuild. GitHub re-runs the failed jobs at the same commit; the skip check covers a full re-run. The workflow grants the token [nothing by default](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#permissions) and each job asks for what it uses: only the publish jobs mint OIDC tokens, and only the tag job pushes. The build runs every dependency's scripts with neither, and the checkouts that never push set `persist-credentials: false`. ## Finish the version or burn it A rerun that skips a landed artifact trusts it, and the digest comparison earns that trust. When the registry holds `1.4.0` with other bytes, the run stops and says so. ```bash node burn.mjs ``` ```text output stopped: npm holds @acme/sdk 1.4.0 with different bytes decision: burn 1.4.0, release 1.4.1 to all six upload attempts after the stop: npm 0, pypi 0, rubygems 0, crates 0 ``` Finish the version when every landed artifact matches what you built, because the registries allow nothing else. npm says a published name and version [can never be used again](https://docs.npmjs.com/cli/v11/commands/npm-publish/), and PyPI does not allow a filename to be reused. Cargo calls a publish [generally permanent](https://doc.rust-lang.org/cargo/reference/publishing.html), and Maven Central says a published component [cannot be changed](https://central.sonatype.org/faq/can-i-change-a-component/). Burn the version when any landed artifact is wrong, or when you cannot tell. Burning means `1.4.1` on all six registries, including the three that hold a correct `1.4.0`, because consumers of a lockstep SDK expect one number everywhere. ## Check it worked Run the suite. Fourteen tests pin the two runs, the burn, the tag, each refusal while a read lags, and the workflow's permissions. ```bash node --test release.test.mjs ``` ```text output 1..14 # tests 14 # suites 0 # pass 14 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 375.184785 ``` `# pass 14` is the line to look for. The fourth test is the one to keep if you cut the rest: the upload meets PyPI's refusal for other bytes, and the run stops without the tag. ## When it goes wrong The tag went out before the uploads. Go and PHP consumers resolve the version the moment the tag exists, so a failure after the tag leaves them with a release the other registries deny: ```bash node pitfall.mjs ``` ```text output tag first, then the uploads tag v1.4.0 pushed: goproxy and packagist now serve 1.4.0 npm @acme/sdk 1.4.0 published pypi acme-sdk 1.4.0 published rubygems acme_sdk 1.4.0 published stopped: crates: acme-sdk 1.4.0 failed, 503 Service Unavailable serving 1.4.0: npm yes, pypi yes, rubygems yes, crates no, goproxy yes, packagist yes v1.4.0 exists: true ``` Push the tag from a job that needs every publish job, as `release.yml` does. The module mirror's own FAQ says a version requested before its tag was pushed can take up to 30 minutes to appear, because the mirror caches the miss.[^3] The probe says absent, but the upload says already there. Packagist's API documentation admits that mirroring delays can hand you an outdated file, and other registries cache their read paths too. On PyPI that refusal means the bytes differ, so `release.mjs` reads again and compares, and stops without the tag while the read still lags. The rerun rebuilt the artifacts and the digests differ. Tarballs, wheels, and gems carry file timestamps, so a rebuild on another day produces different bytes for the same source, and the burn verdict fires on a release that is fine. Publish the artifacts the first run built, as the shared `dist` artifact does, or make the build reproducible; [`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/) is the convention for pinning the timestamps a build embeds. ## When not to do this Do not run this against Maven Central as though it were one of the other five. The Central Portal takes one bundle, validates it as a unit, and either publishes all of it or marks the deployment `FAILED`, so a rerun there is a fresh deployment. The states its [Publisher API](https://central.sonatype.org/publish/publish-portal-api/) exposes, `PENDING`, `VALIDATING`, `VALIDATED`, `PUBLISHING`, `PUBLISHED`, and `FAILED`, are the transaction the other registries lack. Do not treat the skip check as permission to move a tag. For Go and Packagist a tag is the release, so a moved tag is a second release wearing the first one's name. Do not burn a version to avoid the digest comparison. A burned number stays in every registry's history without ever becoming installable, and it needs a changelog entry that says so. Do not skip on existence alone. Skipping without the digest is how a wrong tarball survives the rerun meant to fix the release. ## Related how-tos - [Choose a release automation tool for an SDK repository](/howto/choose-a-release-automation-tool-for-an-sdk) - [Make queue consumer writes safe under redelivery](/howto/queue-consumer-writes-under-at-least-once) ## Last verified Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed. The six registries are a stand-in on 127.0.0.1 whose paths, status codes, and refusal texts follow the documentation and source cited on this page, so no request reached a real registry. `release.yml` was read as a file rather than run on GitHub Actions, and the stand-in's npm refusal text is the message the npm client prints, which no npm document states. The stand-in's RubyGems.org keeps no API keys, so it answers every identical repeat as one long-lived key would. [^1]: Warehouse puts the refusal in the HTTP status line. The function that does it, [`_exc_with_message`](https://github.com/pypi/warehouse/blob/main/warehouse/forklift/utils.py), carries a comment explaining that the old PyPI API used the status to pass messages down to the client. [RFC 9112](https://www.rfc-editor.org/rfc/rfc9112.html#name-status-line) says a client SHOULD ignore the reason-phrase content because it is not a reliable channel for information. [twine](https://github.com/pypa/twine/blob/main/twine/commands/upload.py) reads it regardless, and the comment beside the message in Warehouse warns that changing its first words would break `--skip-existing`. Both projects keep open a channel the protocol asked clients to ignore, with a note on each side saying why. [^2]: RubyGems.org forgives a repeat only from the key that made it. [`pusher.rb`](https://github.com/rubygems/rubygems.org/blob/master/app/models/pusher.rb) looks the gem up by its number, its SHA-256, and the API key that pushed it, so the same bytes from the same key get a 200 reading `Gem was already pushed`. A [trusted publisher](https://guides.rubygems.org/trusted-publishing/) mints a fresh key for each run, so a rerun's identical push gets the 409 meant for different bytes. [^3]: proxy.golang.org is the one registry on this page that the consumer asks rather than the publisher. Its [FAQ](https://proxy.golang.org/) supplies the only duration here: a version requested before its tag existed may take up to 30 minutes to appear, because the mirror caches the miss. The mirror learns of a release from whoever asks first, and it remembers the question.