How-to › Model, configure and generate

How to test a project generator without writing files to disk#

Test a jostraca generator against an in-memory volume, assert on the tree it would write, and keep one smoke test on disk for the reads memory cannot see.

Audience
Library maintainer
Level
intermediate
Topic
Generate code and projects
Languages
TypeScript and Python
Verified

Each test of your generator shells out to the command, writes a folder under the system temp directory, and diffs it against a snapshot. Every case starts a process, and two tests that pick one path overwrite each other. Moved into memory, the tests can pass on your machine and fail on a clean one, because the generator reads a template the test never handed it.

What you get

You will end up with in-memory unit tests for a jostraca generator that assert on the tree it would write. One smoke test runs the command on disk from an empty folder. This is for you if you maintain a generator and its tests are slow or depend on where they start.

Short answer

Run the jostraca generator with Jostraca({ mem: true }) and an absolute output folder, seed its templates through vol, then assert on res.vol().toJSON(). Make one instance per test, because a global mem: true keeps one volume across calls. Keep one smoke test that runs the command on disk from an empty folder and compares a committed tree, which catches a generator that reads the disk itself.

You will need

Node 22 or later, jostraca 0.42.0 with its peer shape 12.2.5, and a generator you already run from the command line. The sample here is a component generator: one model in, a folder of three TypeScript files out, rendered from two templates. Its tests use the Node test runner, so the sample has no other dependency.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
jostraca mem and checkA generator written as a jostraca component treeReads made by your own code reach the real disk, so one smoke test staysThe generator is built on another framework
Nx createTreeWithEmptyWorkspaceGenerators for an Nx workspace, tested against its virtual treegenerateFiles reads templates from the disk, and the empty workspace brings four filesYou are not inside an Nx workspace
OpenAPI Generator —dry-runYou customize its templates and want the file list a run would writeThe report lists paths and states, never contents, and omitted .openapi-generator/FILES hereYou need to assert on generated text
pytest-cookiesA Cookiecutter template tested from pytestEvery bake writes a real folder under pytest’s temporary directoryThe generator is not a Cookiecutter template
Swagger CodegenYou generate with its 3.x line and snapshot the outputIts generate help lists no dry-run option, so every check writes a folderYou want a report without a write
yeoman-testA Yeoman generator with prompts and composed sub-generatorsEach run changes the working directory to a temporary folder on diskTests must run in parallel in one process

Voxgig maintains jostraca, the subject of this page; the other five rows are other projects’ tools.

jostraca and Nx keep the writes in memory, which makes a test fast and leaves nothing behind. Neither proves the files land on disk, and Nx’s generateFiles reads its templates from disk.1 pytest-cookies and yeoman-test write real folders, so they cost a temporary directory per run and prove the write. OpenAPI Generator reports without writing, and Swagger Codegen offers no such mode.

Run the generator against a volume

Write the generator as a function that returns the define callback, so a test and the command call the same code. It reads its templates only through jostraca’s Fragment:

export function componentGenerator(model, { templates = TEMPLATES } = {}) {
  validateModel(model)
  return () => {
    Project({ folder: model.name }, () => {
      File({ name: `${model.name}.tsx` }, () => {
        Fragment({ from: join(templates, 'component.tsx') }, () => {
          Slot({ name: 'props' }, () => {
            Content({ src: model.props.map((p) => `  ${p.name}: ${p.type}`).join('\n'), raw: true })
          })
        })
      })
      File({ name: 'index.ts' }, () => {
        Content({ src: `export { ${model.name} } from './${model.name}'\n`, raw: true })
      })
      File({ name: `${model.name}.test.tsx` }, () => {
        Fragment({ from: join(templates, 'component.test.tsx') })
      })
    })
  }
}

validateModel refuses a name or a type that could end a line or a block. jostraca writes a model value into the output as it is, quotes and backticks included, and a test holds it to that.

Seed the templates into the volume under /tpl, point the generator there, and use an absolute output folder. A relative one resolves against the working directory, so the volume’s keys would carry whatever path the test started in.

test('writes the three files of a component, in memory', async () => {
  const res = await generate(model, { folder: '/out', mem: true, vol: seedTemplates(), templates: MEM_TEMPLATES })
  const files = outputFiles(res, '/out')
  assert.deepEqual(Object.keys(files).sort(), FILES)
  assert.equal(files['OrderList/index.ts'], "export { OrderList } from './OrderList'\n")
  assert.match(files['OrderList/OrderList.tsx'], /^export interface OrderListProps \{\n  orders: Order\[\]\n  onSelect: \(id: string\) => void\n\}\n/)
  assert.ok(!files['OrderList/OrderList.test.tsx'].includes('$$'), 'every $$name$$ was filled in')

outputFiles drops jostraca’s .jostraca/ bookkeeping and the seeded templates before the assertion, because both change when jostraca’s own records change.

Build a fresh Jostraca instance in each test. With mem: true set on the instance and no vol on the call, every generate reuses one volume, which the option handling in 0.42.0 decides. A test that reused one instance saw the first component’s files beside the second’s.2

Keep one smoke test on disk

Run the real command, in a child process, from an empty working directory, and compare every file with a committed copy in expected/. Then run check() over that copy, which generates in memory, reads templates from disk, and reports drift.

test('end to end: the command writes the committed files from an empty folder', async (t) => {
  const out = join(scratch(t), 'out')
  await run(process.execPath, [join(HERE, 'cli.mjs'), 'generate', out], { cwd: scratch(t) })
  assert.deepEqual(tree(out), FILES)
  assert.deepEqual(tree(EXPECTED), FILES)
  for (const f of FILES) {
    assert.ok(readFileSync(join(out, f)).equals(readFileSync(join(EXPECTED, f))), `${f} matches expected/`)
  }
})

The second smoke test edits one file of a copy and deletes another, and check() reports content OrderList/index.ts and missing OrderList/OrderList.test.tsx.

Check it worked

Run the suite, from the sample’s folder:

node --test generator.test.mjs
1..11
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0

The in-memory tests assert on the volume alone. The second test points a memory run at a real folder under the temp directory and asserts that existsSync(folder) is false afterwards.

Preview a template change in the two SDK generators

A team that generates its SDK with OpenAPI Generator tests a template change with generate --dry-run, which reports what a run would write without writing it. Swagger Codegen’s 3.x generate lists no such option, so a team on it generates into a temporary folder and diffs a committed snapshot, the shape of this page’s smoke test. record.sh ran both jars once on the Bookshelf document, with typescript-fetch, and the test helpers from the leaf’s other ecosystems:

node compare.mjs
OpenAPI Generator 7.27.0, generate --dry-run
  exit 0, output folder created: no
  report printed 2 times at ERROR level, 10 paths, states: w
  a real run wrote 11 files; not in the report: .openapi-generator/FILES
Swagger Codegen 3.0.85, generate --dry-run
  exit 0, output folder created: no, printed its usage text: true
  options in generate --help that mention dry: 0; a real run wrote 9 files
python 3.13 | pytest 9.2.0 | pytest-cookies 0.7.0 | cookiecutter 2.7.1
  cookies.bake wrote to disk under pytest's temporary folder: true; tests passed: 2
@nx/devkit 24.0.6 | nx 24.0.6
  tree root /virtual, on disk: false; created in the tree: libs/ui/src/CustomerCard/CustomerCard.tsx
  generateFiles read its templates from disk: true
yeoman-test 11.8.2 | yeoman-generator 10.3.1
  changed the working directory: true, ran under the temp folder: true, file on disk: true

OpenAPI Generator’s dry run is a list of paths with a state letter, logged on standard error at ERROR level, twice in 7.27.0. It cannot show that a template emits the wrong line, so a content test still needs a real run into a temporary folder. Swagger Codegen answered --dry-run with its usage text and exit 0, so a script that reads only its exit status takes the unknown flag for a dry run that passed.

When it goes wrong

The in-memory test passes on your machine and the command fails for a user. The generator reads a file with node:fs in its own define code, which jostraca never sees, so memory mode cannot redirect it. leaky.mjs reads the test template relative to the working directory:

node pitfall.mjs
in memory, started in the sample folder
  leaky generator: 3 files
  fixed generator: 3 files
in memory, started in an empty folder
  leaky generator: ENOENT: no such file or directory, open '<cwd>/templates/component.test.tsx'
  fixed generator: 3 files
jostraca Fragment, in memory, template on disk but not seeded
  refused: check "From" failed (threw: ENOENT: no such file or directory, stat '<templates>/component.tsx')

jostraca’s own components do not read the disk in memory mode: Fragment checks its from path against the run’s file system (line 46 of Fragment.ts), so an unseeded template fails at once. Read every input through a component, or pass it in from the caller, and let the smoke test catch the rest.

check() fails with ENOENT when the templates sit inside the folder it checks. It serves every path under that folder from memory, so keep expected/ outside the generator’s own files.

A dry run in a tool built on jostraca can still write. Both of these were recorded by record.sh:

node compare.mjs dry-runs
@voxgig/model 12.0.2, voxgig-model --dryrun, one action writing twice
  model/model.json: not written
  generated/via-build-fs.txt: not written
  generated/via-node-fs.txt: on disk
@voxgig/sdkgen 4.44.3, npm run dry-generate: exit 0
  ts/src/BookshelfSDK.ts, deleted first: written again
  ts/src/Config.ts, edited first: edit overwritten
@voxgig/sdkgen 4.45.0, npm run dry-generate: exit 0
  ts/src/BookshelfSDK.ts, deleted first: written again
  ts/src/Config.ts, edited first: edit overwritten

The model’s --dryrun swaps the writers of build.fs for an in-memory volume (model.ts lines 47 to 51), so an action that calls node:fs writes to disk: the leaky generator’s fault, one layer up. sdkgen’s build constructs SdkGen from its own options, the logger, and the debug level (sdkgen.ts lines 657 to 661). It leaves out build.fs and the dry-run flag, so the SDK is written, on 4.45.0 as well. Neither is a fault of jostraca’s memory mode.

When not to do this

Do not drop the end-to-end run. A memory test proves the tree jostraca would write, not that the bytes reach the disk, the permissions hold, or the command finds its templates from where a user starts it. One smoke test per command covers that.

Do not port a Cookiecutter template to jostraca to get memory tests. pytest-cookies already bakes into pytest’s temporary folder, and its tests read as plain pytest.3 The same goes for a Yeoman generator with a working yeoman-test suite, unless tests that change the working directory are blocking parallel runs.

Do not reach past jostraca’s own components in a generator you test in memory. If the define code needs a file, read it in the caller and pass it in, or seed it.

Your SDK may come from OpenAPI Generator, and the question may be which files a template change adds or skips. Its --dry-run answers that without a line of test code, which beats writing a jostraca harness. A team on Swagger Codegen’s 3.x line needs nothing from jostraca either: a temporary folder and a diff against committed output is this page’s smoke test, and it suffices.

Last verified

Verified 2026-11-22 against Node 22.23.5, jostraca 0.42.0 and shape 12.2.5. Every output block is what the command before it printed. OpenAPI Generator 7.27.0, Swagger Codegen 3.0.85, Nx 24.0.6, yeoman-test 11.8.2, and pytest-cookies 0.7.0 with Cookiecutter 2.7.1 on Python 3.13 ran once in record.sh. So did @voxgig/model 12.0.2, and sdkgen 4.44.3 and 4.45.0 from a create-sdkgen 0.31.9 scaffold; compare.mjs reads what each recorded.

Footnotes

  1. Nx roots the tree from createTreeWithEmptyWorkspace at /virtual, in one line of its testing utilities. The empty workspace is not quite empty: the run here found nx.json, package.json, tsconfig.base.json, and .oxfmtrc.json waiting in it before the generator wrote a line. ↩︎ Back to text

  2. The Go port of jostraca accepted the memory options for some time before it honored them. The generate-in-memory page says that before v0.35.0 a Go generator given WithMem() and WithVol() wrote to the real file system and returned nil for Vol and FS, with no error, and gives the matter one sentence. ↩︎ Back to text

  3. The pytest-cookies getting started page still asserts on result.project.basename and result.project.isdir(). Release 0.7.0 answers that property with a DeprecationWarning. Its text, in the plugin’s source, says the property “is deprecated and will be removed in a future release” and asks for project_path in its place. The documented first test is the one that warns. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.