Your release job loops over 600 generated repositories and publishes each package in turn. At repository 212 a manifest is truncated, the loop throws, and the other 388 wait for the next scheduled run. When it runs again, the first 211 are refused as published and the job fails at the first of them.
What you get
You will end up with a fan-out split into shards that publishes each repository independently, paces itself under a registry limit, and writes a status page. This is for you if one pipeline releases the packages of many generated repositories.
Short answer
Sort the repositories and give each CI job one shard of the list. In each shard, check the package’s name, version, and disclaimer, ask the registry whether those bytes are already there, and publish under a pace that divides the registry budget between shards. Record every repository as published, skipped, or failed, and keep going. A rerun then publishes only what is missing.
You will need
Node 22 or later with npm 10, and a list of generated repositories that each hold a package. Run
Release six SDK packages from one commit and
Recover when one registry upload fails mid release
first: this page repeats their per-repository steps hundreds of times. The sample publishes to a
registry stand-in on 127.0.0.1 with the real npm publish, and needs no account. Only the
TypeScript package of each fixture is published: the other five languages appear in the registry
scan alone.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| GitHub Actions reusable workflows with multi-gitter | Each repository releases itself, and one change must reach every repository | A pull request per repository to roll out, and no single view of what released | One pipeline should own the release |
| Kiota | Many clients regenerated from their lock files with kiota update | No publishing at all, and kiota update 1.38.0 exited 0 over an unreadable description | You need the tool to report a failed client |
| OpenAPI Generator batch | Many clients generated in one run from one config per client | Generates only; publishing and status are still your loop | Your clients come from different generators |
| per-language monorepos as in the Azure and AWS SDKs | You own the APIs and want one repository per language | Each client’s history, issues, and releases share one repository with every other service | Each client has its own upstream and its own cadence |
| SDK Catalog fan-out | Hundreds of generated repositories, one per API, each with several language targets | One repository per API to keep; the scan found Go modules for 636 and two npm packages | A handful of SDKs that fit in one repository |
Voxgig maintains the SDK Catalog, which is the subject of this page. Kiota, OpenAPI Generator, multi-gitter, and the two monorepos are other projects’.
A monorepo such as aws-sdk-js-v3, with a separate package per service, keeps every release in one place and ties them together. A fan-out keeps each repository separate, so isolation is cheap and the status page has to be built. multi-gitter changes many repositories but publishes none of them itself.
Know what the catalog has published
record.sh asked each registry about every repository the site lists. The record is one day’s
reading:
cat records/registries.txt
scanned on 2026-11-22
repositories the site lists: 637
npm: 2 (learnworlds, voxgig-solardemo)
PyPI: 0
RubyGems: 0
Go module proxy: 636
Packagist vendor voxgig-sdk: 0 packages
LuaRocks user voxgig-sdk: status 404
The Go module proxy serves a tagged module straight from its
repository, with no upload step, which is why 636 resolve. The two npm packages use the short name
without -sdk. No catalog package is on PyPI, Packagist, RubyGems, or LuaRocks.
Shard the list and isolate each repository
Sort the list, so shard k of n is the same repositories on every machine:
export function shard(repos, index, total) {
if (!Number.isInteger(total) || total < 1 || !Number.isInteger(index) || index < 1 || index > total) {
throw new Error(`bad shard ${index}/${total}`)
}
return repos.filter((_, i) => i % total === index - 1)
}
GitHub Actions caps a job matrix at 256 jobs per run and a hosted job at six hours, as its
limits page states,1 so 637 repositories
need fewer shards than repositories. publishShard catches every error per repository and records
it. preflight refuses a package whose name is outside @voxgig-sdk/, and one whose README
does not match UNAFFILIATED, the pattern for the phrase not affiliated with.
Split the registry’s budget between the shards, with headroom for npm’s start-up time:
const SHARDS = 3
const BUDGET = { limit: 3, windowMs: 1000 } // the stand-in's limit, shared by every shard
// Each shard takes a third of the budget, over a window half again as long, so
// the time npm takes to start cannot bunch two publishes into one window.
const PER_SHARD = { limit: Math.max(1, Math.floor(BUDGET.limit / SHARDS)), windowMs: BUDGET.windowMs * 1.5 }
The limit is the stand-in’s own, because npm documents no publish rate. A 429 that gets past the
pace is retried after a growing wait of the loop’s own, not the Retry-After value that
RFC 6585 lets a server send.
Run the fan-out twice
demo.mjs makes nine fixture repositories named after catalog entries, cuts one manifest short,
strips one README of its disclaimer, and runs three shards. Then it restores the README and
runs again:
node demo.mjs
run 1
| repository | shard | package | result | note |
| --- | --- | --- | --- | --- |
| abhi-sdk | 1/3 | @voxgig-sdk/abhi-sdk@0.0.1 | published | |
| california-state-portal-sdk | 2/3 | @voxgig-sdk/california-state-portal-sdk@0.0.1 | published | |
| domain-whois-sdk | 3/3 | @voxgig-sdk/domain-whois-sdk@0.0.1 | published | |
| free-uv-index-sdk | 1/3 | @voxgig-sdk/free-uv-index-sdk@0.0.1 | published | |
| ip-intelligence-api2-sdk | 2/3 | @voxgig-sdk/ip-intelligence-api2-sdk@0.0.1 | published | |
| opensanctum-sdk | 3/3 | - | failed | package.json is not valid JSON |
| rick-and-morty-sdk | 1/3 | @voxgig-sdk/rick-and-morty-sdk@0.0.1 | published | |
| top-news-sdk | 2/3 | - | failed | README does not say the client is unaffiliated |
| yugi-limit-regulation-sdk | 3/3 | @voxgig-sdk/yugi-limit-regulation-sdk@0.0.1 | published | |
published 7, skipped 0, failed 2
run 2, after the top-news-sdk README is fixed
| repository | shard | package | result | note |
| --- | --- | --- | --- | --- |
| abhi-sdk | 1/3 | @voxgig-sdk/abhi-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| california-state-portal-sdk | 2/3 | @voxgig-sdk/california-state-portal-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| domain-whois-sdk | 3/3 | @voxgig-sdk/domain-whois-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| free-uv-index-sdk | 1/3 | @voxgig-sdk/free-uv-index-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| ip-intelligence-api2-sdk | 2/3 | @voxgig-sdk/ip-intelligence-api2-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| opensanctum-sdk | 3/3 | - | failed | package.json is not valid JSON |
| rick-and-morty-sdk | 1/3 | @voxgig-sdk/rick-and-morty-sdk@0.0.1 | skipped | already on the registry with the same bytes |
| top-news-sdk | 2/3 | @voxgig-sdk/top-news-sdk@0.0.1 | published | |
| yugi-limit-regulation-sdk | 3/3 | @voxgig-sdk/yugi-limit-regulation-sdk@0.0.1 | skipped | already on the registry with the same bytes |
published 1, skipped 7, failed 1
registry: 8 packages, 8 at 0.0.1
The broken manifest failed alone, and its shard went on to yugi-limit-regulation-sdk. The
second run compared each package’s npm pack integrity with the registry’s and skipped the seven
that matched. Commit the table as the status page, because a registry lists what arrived and
says nothing about what failed.
Fan out with Kiota and OpenAPI Generator
Neither tool publishes, so each team keeps a loop like publishShard and changes only the
generate step. record.sh gave both three clients, one from a description cut off mid-file:
cat records/generators.txt
OpenAPI Generator 7.27.0, batch over three configs, broken.json first
batch:
exit 1, [FAIL] Completed with 1 failures, 2 successes
written: quotes-sdk tides-sdk
batch --fail-fast:
exit 1, no summary line
written:
Kiota 1.38.0, three clients generated, then broken.json cut off and kiota update -o repos
exit 0
fail: Kiota.Builder.KiotaBuilder[49154641]
Update of 3 clients completed successfully
OpenAPI error reported: 1
broken-sdk files the update changed: repos/broken-sdk/ts/.kiota.log repos/broken-sdk/ts/kiota-lock.json
broken-sdk files before and after: 4 and 5
OpenAPI Generator’s batch isolates the bad config unless you pass --fail-fast, and its exit
status says one failed. Kiota’s
kiota update refreshes every client
under a folder from its lock file, and here it reported success over a client it could not
regenerate. It rewrote that client’s lock file and added a log, and left its code alone. Grep its output for fail: per client, and do not trust the exit status of that
release.
Check it worked
node --test fanout.test.mjs
1..12
# tests 12
# suites 0
# pass 12
# fail 0
# cancelled 0
# skipped 0
# todo 0
The test that carries the page asserts the two status tables, and that the registry holds eight
packages and not opensanctum-sdk.
When it goes wrong
A rerun fails on everything already published.
npm publish fails when the name and
version exist. Read the registry before publishing, and treat a refusal as published only when
the integrity matches, which publishShard does after an E403.
A regenerated package keeps its version. Its bytes then differ from the registry’s, and a test
shows publishShard failing it rather than skipping it. Bump the version in the generator.
A package publishes somewhere else. An .npmrc inside the package can name a registry for the
@voxgig-sdk scope, which beats --registry, so publishShard passes the scoped registry too and
preflight refuses a package that carries one. A test shows nothing reaching a second registry.
The shards hit 429 together. Each shard paced itself to the full budget. Divide the budget by the
shard count, as PER_SHARD does.
When not to do this
Do not fan out a handful of SDKs. Below a few dozen repositories, the per-language monorepo or a loop in one job gives one release history and needs no status page.
Do not publish under a name that implies the API owner made the client. The catalog scopes every
package under @voxgig-sdk/ and says in its READMEs that it is not affiliated, and preflight
refuses a package that does neither.
Do not treat the SDK Catalog’s fan-out as proven for Python, PHP, Ruby, or Lua: none of those
registries holds a catalog package. Prefer OpenAPI Generator’s batch when one tool generates
every client, because it reports failures in its exit status. Prefer Kiota when the clients already
carry lock files, so one command refreshes them all.
Related how-tos
Last verified
Verified 2026-11-22 against Node 22.23.5 and npm 10.9.9, publishing to a stand-in rather than to
npm. record.sh queried npm, PyPI, RubyGems, Packagist, LuaRocks, and the Go module proxy for all
637 listed repositories, and ran OpenAPI Generator 7.27.0 and Kiota 1.38.0. Kiota ran once, with its
telemetry left on, and was not run again; its 1.39.0 release was not tested. A second run of
everything else in record.sh gave the same records.
Footnotes
-
The same limits page gives a workflow run 35 days in all. A release of 637 repositories is therefore bounded by the six hours each job gets rather than by the run. GitHub states both limits for hosted runners and gives a self-hosted job five days. ↩︎ Back to text