How-to › Parse, validate and transform data

How to include one config file from another#

Pull a shared fragment into a config file with @tabnas/multisource, track where each value came from, and refuse cycles and paths that leave the config directory.

Audience
Platform team
Level
advanced
Topic
Parse config and data formats
Languages
TypeScript
Verified

Read first: Accept comments and unquoted keys in a JSON config file

Six services carry their own copy of the same database block, so a host change takes six edits, and the service somebody missed still points at the old host. Moving the block into a shared file needs an include, and a resolver that only joins paths follows ../ wherever it leads. A config file can then pull in your secrets file, or include itself until the stack runs out.

What you get

You will end up with config files that include shared fragments and a loader that knows which file each value came from. A cycle or an escaping path is an error that names the file behind it. This is for you if several services share config.

Short answer

Mark the include in the config file, resolve it relative to the file that holds it, and merge the fragment before that file’s own keys so the including file wins. With @tabnas/multisource the mark is @"shared/db.jsonic", and a cycle is already an error. A path is not: the default file resolver follows ../ anywhere and the default processor runs a .js include, so supply a resolver that keeps every include inside the config directory and counts them.

You will need

Node 22.18 or later, and Python 3.8 or later only for the Jsonnet comparison. Verified 2026-10-15 against Node 22.22.0, @tabnas/multisource 0.6.3, @tabnas/directive 0.5.13, @tabnas/jsonic 0.7.9, @tabnas/parser 0.12.11, js-yaml 5.4.1 and Jsonnet 0.22.0. The @tabnas packages declare Node 24 in engines, so npm warns on Node 22, and everything here ran on 22. The page builds on accepting comments and unquoted keys in a JSON config.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
@tabnas/multisourceConfig in jsonic, where @"file" should splice a fragment in and merge itPaths are unconfined, .js includes run, and nothing caps the number of includes until you supply your own resolverOther tools must read the same files as JSON or YAML
hand-rolled resolver over jsonicYou need the file each value came from, or include rules of your ownA loader you secure yourself: cycles, paths, links, and the number of includesA library already merges the way you want
Jsonnet importConfig built with functions and conditionals, not only shared fragmentsA language every config author learns, and imports that read any file by defaultThe config is data people edit, not a program
YAML anchors and merge keysRepeated blocks inside one YAML fileOne document only, a one-level merge, and js-yaml 5 leaves it off by defaultThe shared block lives in a second file

Voxgig maintains tabnas and jsonic. This page compares @tabnas/multisource and a loader built on jsonic with Jsonnet import and YAML anchors.

YAML anchors cost nothing to adopt and stop at the edge of the file. Jsonnet crosses files and adds functions, at the price of a language. multisource and the hand-rolled loader sit between them: includes in a data format, with the safety of every include path left to you.

Mark the include in the config file

@ followed by a path splices that file in. At the top of a map it merges the file’s keys into the map, before the keys written after it.

# The service's config. Shared settings come first, so keys below override them.
@"shared/defaults.jsonic"
name: orders-api
db: {
  @"shared/db.jsonic"
  pool: { max: 20 }
}

multisource is a plugin for the tabnas engine, built on @tabnas/directive, which lets a token such as @ trigger parsing of its own. The resolver is the part that finds the file, and it is a function you can replace.

Confine every include

The rule fits in one function: an include is a relative path to a .jsonic file, and it must land inside the config directory after links are followed.

  function confine(from: string, include: unknown, where = name(from)): string {
    const refuse = (why: string): never => {
      throw new IncludeError(name(from), `${where}: include ${JSON.stringify(include)} ${why}`)
    }
    if (typeof include !== 'string' || include === '') return refuse('is not a path')
    if (isAbsolute(include)) return refuse('is an absolute path')
    if (extname(include) !== '.jsonic') return refuse('is not a .jsonic file')
    // Refuse on the path as written before touching the filesystem, so the
    // error cannot be used to learn which files exist outside the directory.
    const full = resolve(dirname(from), include)
    if (!inside(full)) return refuse(`is outside ${name(top)}`)
    let real = ''
    try {
      real = realpathSync(full)
    } catch {}
    // A link inside the directory can point anywhere. A file reached through
    // one gets the same answer as a missing file, or a committed link would
    // tell the config's author which files exist outside.
    if (!inside(real)) return refuse(`was not found in ${name(top)}`)
    return real
  }

The lexical check comes first, so the error for an include such as ../x.jsonic is the same whether that file exists or not. realpathSync then follows links, which is what catches a link inside the directory that points out of it. A file reached that way gets the same “was not found” as a missing file, so a link committed into the directory cannot probe the disk either. On Linux a backslash and %2e are ordinary characters, so ..\secrets.jsonic names a missing file inside the directory, and a test pins both spellings.

The multisource resolver wraps confine, adds the line and column of the @ token to the message, and counts includes:

export function confinedResolver(root: string): Resolver {
  const { confine, name } = makeConfine(root)
  let includes = 0
  return (spec, popts, rule, ctx) => {
    // The file holding the include, and the @ token that starts it.
    const from: string = ctx.meta?.multisource?.path ?? popts.path!
    const at = rule.parent?.o0
    const where = `${name(from)}:${at?.rI}:${at?.cI}`
    if (++includes > MAX_INCLUDES) throw new IncludeError(name(from), `${where}: more than ${MAX_INCLUDES} includes`)
    const path = typeof spec === 'string' ? spec : spec.path
    const full = confine(from, path, where)
    return { kind: 'jsonic', path, full, abs: false, src: readFileSync(full, 'utf8'), found: true }
  }
}

Four files through the loader: one with two includes, a cycle, an include of ../secrets.jsonic, and an include of ../payload.js.

node demo.ts
config/app.jsonic
  {"log":{"level":"info"},"port":8080,"name":"orders-api","db":{"host":"db.internal","pool":{"min":2,"max":20}}}
  config/app.jsonic includes config/shared/defaults.jsonic
  config/app.jsonic includes config/shared/db.jsonic

config/cycle/a.jsonic
  SyntaxError: [jsonic/multisource_cycle]: source includes itself: config/cycle/a.jsonic
    --> b.jsonic:1:1
    1 | @"a.jsonic"
        ^ source includes itself: config/cycle/a.jsonic
    2 | y: 1
    3 |
    Including config/cycle/a.jsonic here would loop forever:
    config/cycle/a.jsonic -> config/cycle/b.jsonic -> config/cycle/a.jsonic

config/escape.jsonic
  IncludeError: config/escape.jsonic:2:1: include "../secrets.jsonic" is outside config

config/runs.jsonic
  IncludeError: config/runs.jsonic:2:1: include "../payload.js" is not a .jsonic file

db.pool kept min: 2 from the shared file and took max: 20 from app.jsonic, a deep merge in which the including file wins. The cycle error comes from multisource itself. It compares each include against the chain of files that led to it, so a file included from two branches is reuse, and only an ancestor is a loop. The include graph comes back in deps.

Keep the origin of every value

deps says which file included which, not where db.pool.max came from. For that the merge has to be yours. This loader parses each file with jsonic, replaces a $include key with the files it names, and carries the source file on every leaf until the end.

  function load(full: string, chain: string[]): Tree {
    if (chain.includes(full)) {
      const by = name(chain[chain.length - 1])
      const loop = [...chain.slice(chain.indexOf(full)), full].map(name).join(' -> ')
      throw new IncludeError(by, `${by}: include loops back on itself: ${loop}`)
    }
    if (++loads > MAX_INCLUDES) throw new IncludeError(name(full), `more than ${MAX_INCLUDES} includes`)
    // fileName makes a syntax error in this file name this file.
    const parsed = Jsonic(readFileSync(full, 'utf8'), { fileName: name(full) })
    return expand(parsed, full, [...chain, full])
  }

The chain check is the same one multisource makes. multisource sets no limit on the number of includes, and it parses a file again each time it is included. A file included twice by a file included twice, twenty levels down, is then a million loads from twenty-one small files.1 Both loaders here stop at MAX_INCLUDES, which is 1000: own.ts counts files loaded, and confinedResolver counts includes resolved. The merge order differs too. multisource merges in the order the file is written, so an include placed after a key overrides it, but own.ts always lets the including file win.

node origins.ts
config/own/app.jsonic
  {"log":{"level":"info"},"port":8080,"name":"orders-api","db":{"host":"db.internal","pool":{"min":2,"max":20}}}
  log.level    config/shared/defaults.jsonic
  port         config/shared/defaults.jsonic
  name         config/own/app.jsonic
  db.host      config/shared/db.jsonic
  db.pool.min  config/shared/db.jsonic
  db.pool.max  config/own/app.jsonic

config/own/cycle-a.jsonic
  IncludeError: config/own/cycle-b.jsonic: include loops back on itself: config/own/cycle-a.jsonic -> config/own/cycle-b.jsonic -> config/own/cycle-a.jsonic

config/own/escape.jsonic
  IncludeError: config/own/escape.jsonic: include "../../secrets.jsonic" is outside config

Compare YAML anchors and a config language

YAML reuses a block through an anchor and an alias, and the merge key << copies a mapping into another. js-yaml 5 leaves the merge key out of its default schema.2

node yaml-anchors.ts
yaml/app.yaml, default schema
  service.db: {"<<":{"host":"db.internal","pool":{"min":2,"max":10}},"pool":{"max":20}}
yaml/app.yaml, CORE_SCHEMA.withTags(mergeTag)
  service.db: {"host":"db.internal","pool":{"max":20}}
yaml/service.yaml, CORE_SCHEMA.withTags(mergeTag)
  YAMLException: unidentified alias "defaults" at 3:8

By default << is an ordinary key. With the merge tag on, the merge is one level deep, so the pool written in the service replaces the shared pool and min: 2 is gone. A second file cannot see the first file’s anchors at all.

Jsonnet resolves an import relative to the importing file, and importstr reads any file as text. The Python binding takes an import_callback, which is where confinement goes:

PYTHONPATH=vendor python3 jsonnet_import.py
default imports
  jsonnet/app.jsonnet  {"db": {"host": "db.internal", "pool": {"max": 20, "min": 2}}, "log": {"level": "info"}, "name": "orders-api", "port": 8080}
  jsonnet/cycle-a.jsonnet  RUNTIME ERROR: max stack frames exceeded.
    jsonnet/cycle-b.jsonnet:1:2-26  thunk <import>
  jsonnet/escape.jsonnet  {"leaked": "# Outside every config directory in this sample. No include may read it.\napiToken: s3cr3t-outside-config\n", "name": "reports"}
import_callback
  jsonnet/app.jsonnet  {"db": {"host": "db.internal", "pool": {"max": 20, "min": 2}}, "log": {"level": "info"}, "name": "orders-api", "port": 8080}
  jsonnet/cycle-a.jsonnet  RUNTIME ERROR: max stack frames exceeded.
    jsonnet/cycle-b.jsonnet:1:2-26  thunk <import>
  jsonnet/escape.jsonnet  RUNTIME ERROR: couldn't open import "../secrets.jsonic": import '../secrets.jsonic' is outside jsonnet/
    jsonnet/escape.jsonnet:2:28-57  object <anonymous>

Jsonnet merges deeply with pool+:, and it names a file in the loop, but as a stack overflow rather than a cycle.3

Check it worked

Fifteen tests cover both loaders with the Node test runner. They cover a link out of the directory, a linked directory used to probe for files, and absolute, backslash, and percent-encoded paths. They also cover a file included from two branches, an include bomb through each loader, and a __proto__ key. The escape test also checks that the secret is not quoted back in the error.

test('an include that leaves the directory is refused, with no secret in the message', () => {
  assert.throws(() => loadConfig('config/escape.jsonic', 'config'), (err: any) => {
    assert.ok(err instanceof IncludeError)
    assert.equal(err.message, 'config/escape.jsonic:2:1: include "../secrets.jsonic" is outside config')
    assert.doesNotMatch(err.message, /s3cr3t/)
    return true
  })
  assert.throws(() => loadWithOrigins('config/own/escape.jsonic', 'config'), {
    message: 'config/own/escape.jsonic: include "../../secrets.jsonic" is outside config',
  })
})
node --test include.test.ts
# tests 15
# suites 0
# pass 15
# fail 0

When it goes wrong

A config file reads a file it should not, or runs code. The same two files through the makeFileResolver that multisource ships:

node pitfall.ts
config/escape.jsonic
  {"name":"reports","apiToken":"s3cr3t-outside-config"}
config/runs.jsonic
payload.js ran inside the config loader
  {"name":"reports","ran":true}

The file resolver in 0.6.3 resolves an include against the including file’s folder, with no root to stay inside. When the file is missing, it tries node_modules folders up to seven levels up the tree. The default processor for a .js include is require, so an include runs whatever the file contains. Pass confinedResolver instead. A test pins the default behavior, so a release that changes it shows up.

When not to do this

Do not use the default multisource resolver on config written by anyone you would not give a shell. It reads any path, and the default processor runs any .js file it is pointed at.

Do not fetch a URL inside the parse. The multisource Resolver is a synchronous function, so a remote fragment means blocking the parse on the network. Download it in a build step, pin it by hash, and include the local copy.

Do not hand-roll a loader unless you need the origin of each value or rules multisource lacks. The hundred-odd lines of own.ts and confine.ts are security code you now maintain.

Do not move YAML config to tabnas to gain includes. If other tools read those files, keep YAML and generate the shared parts, and accept that anchors stop at the file.

Last verified

Verified 2026-10-15 against Node 22.22.0, @tabnas/multisource 0.6.3, @tabnas/directive 0.5.13, @tabnas/jsonic 0.7.9, @tabnas/parser 0.12.11, js-yaml 5.4.1 and Jsonnet 0.22.0 through its Python binding. Every output block is what the command preceding it printed.

Footnotes

  1. The shape is older than config files. CWE-776 records it for XML, where nested entity definitions expand exponentially, and lists three names for it: XEE, Billion Laughs Attack and XML Bomb. The entry notes that XML Bomb was the term in the early years, and that XEE now seems to be the more common one. ↩︎ Back to text

  2. The merge key’s own page at yaml.org describes it as a language-independent type for YAML 1.1, with the status Working Draft and the date 2005-01-18. The YAML 1.2.2 specification does not mention it, so a << in a config file rests on a working draft dated twenty-one years back. ↩︎ Back to text

  3. The Jsonnet bindings documentation offers import_callback for trapping import, importstr and importbin, and gives reading files out of archives and implementing library search paths as its uses. The list does not name keeping a config away from the rest of the disk, although the same hook does that in eight lines of jsonnet_import.py. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.