How to run a coding agent headless in CI to keep a client current#

The upstream API renames a response field, your client sums undefined, and the usage dashboard reads NaN until a customer asks why. An agent in CI could make the fix the day the spec changes. The same agent, unattended, holds your repository token and the network, and the quickest route it has to a green check is to skip the failing test.

What you get

You will end up with a workflow where a read-only job lets an agent fix the client. A job without the model key checks the change, and a third opens the pull request. This is for you if you maintain a client for an API you do not control.

Short answer

Split the workflow by what each job holds. The agent job holds the model key and only contents: read, checks out with persist-credentials: false, runs the agent with a turn limit and a sandbox, and exports the diff. A job without the key guards that patch and runs the tests against a mock. A third, with write access and no key, opens the pull request with a GitHub App token, so required checks gate the merge.

You will need

A GitHub repository with Actions enabled, and a model API key for one agent. A GitHub App that can push branches and open pull requests, and tests that run against a mock rather than the live API. Verified 2026-10-15 against Node 22.22.0 and yaml 2.9.1. The prompt, and the diff of the spec it carries, are a separate task, and so is building the mock.

Approaches compared

The three CLI routes put the prompt, the sandbox flags, and the permissions in a file you review, and leave you to contain them. Copilot’s agent arrives contained: it pushes only to a copilot/ branch, works behind a firewall, and by default its pull requests wait for a person before their workflows run. That containment is the cost as well, because you choose fewer of its tools and its runs stop at that approval unless you switch it off.

Keep the model key out of the job that can write

The split comes from the Codex documentation, which gives the agent job contents: read only and a second job the write permissions without the key. It applies to every route. A detect job fetches the published spec on a weekday schedule¹ and stops the run when it matches the vendored copy. The agent job starts like this:

  agent:
    needs: detect
    if: needs.detect.outputs.changed == 'true'
    runs-on: ubuntu-latest
    timeout-minutes: 30
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

persist-credentials: false stops checkout leaving the job token where every later git command, the agent’s included, can use it. The agent step then fixes the turn limit, the permission mode, and the tools:

      - name: Run the agent
        uses: anthropics/claude-code-action@374f254f5efaf968abfe15d399fd90972e8138b9 # v1.0.251
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          github_token: ${{ github.token }}
          settings: ci/claude-settings.json
          prompt: |
            The upstream OpenAPI document changed. spec/openapi.json is the new version
            and spec-diff.txt shows what changed. Update the client in src/ so that
            `npm test` passes against the mock in test/. Change files under src/ and
            test/ only. Do not skip, delete or weaken a test. If the change cannot be
            made without breaking a caller, stop and explain why in your final message.
          claude_args: >-
            --max-turns 40
            --permission-mode dontAsk
            --allowedTools "Read,Edit,Write,Bash(npm test),Bash(node --test *)"
            --disallowedTools "WebFetch,WebSearch"
      # The agent's edits are code, ci/guard.mjs included. Running the guard or the tests here
      # would run that code outside the sandbox, in the job that holds the key, so this job
      # only exports the diff.
      - name: Export the change as a patch
        run: |
          rm -f spec-diff.txt
          git add -N .
          git diff --binary HEAD > "$RUNNER_TEMP/agent.patch"

github_token is the line that matters most. Without it the action authenticates as the Claude GitHub App, whose permissions do not depend on the job’s. dontAsk denies every call that would prompt, so the agent gets the tools listed and nothing else. ci/claude-settings.json turns the sandbox on with an empty domain allowlist and failIfUnavailable. Under dontAsk the sandbox refuses a shell command’s connection to any host off the list, and a runner that cannot start the sandbox stops the run.

The tests still reach their mock, because test/mock.mjs starts it inside the test command. On Linux each command in the sandbox gets a localhost private to it, so a mock started by an earlier step would be out of reach.

The job stops at the diff, because everything the agent wrote, ci/guard.mjs included, is code a later step would run outside the sandbox beside the key. The Codex example also only serializes the diff after the agent. Its page warns that tests can read a key set in their job’s environment. A verify job without the key guards the patch, applies it, and runs npm test.

The other two CLIs fit the same slot. Codex takes the sandbox as an input:

      - name: Run the agent
        uses: openai/codex-action@bdf19a4a223ec2549a3e2274a0cf61556bc07675 # v1.13
        with:
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
          prompt-file: ci/spec-sync-prompt.md
          permission-profile: ":workspace"
          safety-strategy: drop-sudo

Gemini CLI takes it as settings. Its action passes --yolo on every run, which approves every tool call whatever defaultApprovalMode says. The configuration reference describes security.disableYoloMode as disabling YOLO mode even when a flag enables it:

      - name: Run the agent
        uses: google-github-actions/run-gemini-cli@f77273f4c914e4bf38440cf36a0369cb64a37489 # v0.1.22
        with:
          gemini_api_key: ${{ secrets.GEMINI_API_KEY }}
          prompt: Update src/ for the spec change in spec/openapi.json. Change src/ and test/ only.
          settings: |
            {
              "general": { "defaultApprovalMode": "auto_edit" },
              "security": { "disableYoloMode": true },
              "tools": { "sandbox": "docker", "sandboxNetworkAccess": false }
            }

Copilot has no step to put there. The workflow opens an issue and assigns it to copilot-swe-agent[bot] through the issues API, and the agent runs in its own environment. The documentation describes that call with a personal access token, not the job’s token:

gh api --method POST "/repos/$GITHUB_REPOSITORY/issues" --input - <<'JSON'
{
  "title": "Update the client for the upstream spec change",
  "body": "spec/openapi.json changed upstream. Update src/ so npm test passes. Change src/ and test/ only.",
  "assignees": ["copilot-swe-agent[bot]"]
}
JSON

Guard the patch, not the agent’s account of it

The guard reads the diff and refuses anything outside src/ and test/, a deleted test, a skipped test, a symbolic link, a binary file, or a change over 400 lines. It reads the patch the way git apply will, which takes a plain --- a/package.json header anywhere outside a hunk. So the guard counts each hunk’s lines from its header and refuses any line it does not expect:

    if (oldLeft > 0 || newLeft > 0) {
      if (line.startsWith('+') && newLeft > 0) { cur.added.push(line.slice(1)); newLeft-- }
      else if (line.startsWith('-') && oldLeft > 0) { cur.removed++; oldLeft-- }
      else if ((line.startsWith(' ') || line === '') && oldLeft > 0 && newLeft > 0) { oldLeft--; newLeft-- }
      else if (line !== '\\ No newline at end of file') fail(n, line)
      continue
    }

Each finding then comes from a file record:

  for (const f of files) {
    if (f.unparsed) { findings.push(`cannot read this patch at ${f.from}`); continue }
    // A rename has two paths and both have to be in bounds: moving a test out of test/ deletes it.
    for (const path of new Set([f.from, f.to])) {
      if (!ALLOWED.some((re) => re.test(path))) findings.push(`${path}: outside src/ and test/`)
    }
    if (f.binary) findings.push(`${f.to}: binary change`)
    if (f.symlink) findings.push(`${f.to}: adds a symbolic link`)
    if (/^test\//.test(f.from) && (f.status === 'deleted' || (f.status === 'renamed' && !/^test\//.test(f.to)))) {
      findings.push(`${f.from}: deletes a test file`)
    }
    if (/^test\//.test(f.to)) {
      for (const added of f.added) if (SKIPS.test(added)) findings.push(`${f.to}: skips a test: ${added.trim()}`)
    }
    changed += f.added.length + f.removed
  }

pipeline.mjs runs the agent and verify steps locally against a copy of project/, a client written for spec 1.4.0. It swaps in spec 2.0.0, where kwh became a decimal value and a unit, and then lets the agent edit twice. The agent is stub-agent.mjs, a script with two scripted edits, so no model is called.

node pipeline.mjs

spec 1.4.0, tests: pass 2, fail 0, skipped 0
spec 2.0.0, tests: pass 1, fail 1, skipped 0
run 1
  agent (fix) changed: src/client.mjs
  guard: ok, 7 changed lines
  tests after the agent: pass 2, fail 0, skipped 0
run 2
  agent (cheat) changed: .github/workflows/ci.yml, package.json, test/client.test.mjs
  guard: refused, 3 findings
    .github/workflows/ci.yml: outside src/ and test/
    package.json: outside src/ and test/
    test/client.test.mjs: skips a test: test.skip('sums a meter\'s readings in kWh', async () => {
  tests after the agent: pass 1, fail 0, skipped 1

Read the last line of run 2. The tests pass, with one skipped, so a gate that runs only the tests would let it through. The guard refuses it on all three edits.

Lint the workflow so the split survives edits

GitHub runs a workflow that breaks every rule on this page without complaint, so lint-workflow.mjs checks the file itself. It wants top-level permissions: {} and no write permission in a job that holds a model key. It also wants no key in env, credentials not persisted, actions pinned to a commit, a timeout, and a turn limit. After the agent step, that job may run nothing from the checkout, and a job without the key has to run the guard. Run it on the workflow and on a one-job first draft:

node lint-workflow.mjs workflow/spec-sync.yml fixtures/one-job.yml

ok   workflow/spec-sync.yml
FAIL fixtures/one-job.yml
  workflow: no top-level permissions, so a job without its own block gets the repository default
  workflow: no concurrency group, so two spec changes can race to open two pull requests
  jobs.sync: a model key in job env reaches every step, npm install scripts included
  jobs.sync: actions/checkout@v7 is not pinned to a commit
  jobs.sync: anthropics/claude-code-action@v1 is not pinned to a commit
  jobs.sync: holds a model key and a write permission in the same job
  jobs.sync: runs an agent with no timeout-minutes
  jobs.sync: checkout persists the job token, so every git command the agent runs is authenticated
  jobs.sync: the agent runs with permission checks or the sandbox turned off
  jobs.sync: claude_args sets no --max-turns
  jobs.sync: no github_token, so the action acts as the Claude GitHub App with its own write access
  workflow: no job without the model key guards the patch

The linter finds an agent job by its secret, so it knows the key names it was written for. Add yours to MODEL_KEY if it is called something else. For the Gemini action it also wants security.disableYoloMode, because the action’s --yolo never appears in the workflow file.

Check it worked

The suite builds each patch with real git and checks the linter against fifteen edits to the workflow, each of which must produce exactly one finding. It also holds four hand-written patches that git apply --check accepts and the guard refuses:

npm test

# tests 33
# suites 0
# pass 33
# fail 0

The end-to-end check needs GitHub and a model key, so this page did not run it. In a test repository, point the detect step’s URL at a copy of the spec with one field renamed, then run the workflow and read the result:

gh workflow run spec-sync.yml
RUN=$(gh run list --workflow spec-sync.yml --limit 1 --json databaseId --jq '.[0].databaseId')
gh run watch "$RUN" --exit-status
gh pr diff "agent/spec-sync-$RUN" --name-only
gh pr checks "agent/spec-sync-$RUN" --required --watch

gh pr diff should list spec/openapi.json and files under src/ or test/ only, and gh pr checks should end with every required check passing.

When it goes wrong

The pull request has no checks, only a banner asking someone to approve its runs. It was opened with GITHUB_TOKEN, and a pull request opened with that token creates its runs in an approval-required state.² Open it with a GitHub App token, as open-pr does.

The agent’s shell commands reached the network although the settings turned the sandbox on. Without failIfUnavailable, Claude Code runs commands outside the sandbox when it cannot start, as on a runner without bubblewrap and socat. Install both, keep failIfUnavailable and allowUnsandboxedCommands: false, and add the AppArmor profile, because on Ubuntu 24.04 the default policy stops bubblewrap creating the user namespaces it needs.

A pinned SHA does not match the commit the release points to. For an annotated tag, git ls-remote prints the tag object first and the commit on a second line ending ^{}, and GitHub’s hardening guide asks for the commit. Two of the pins on this page are annotated tags.

When not to do this

Do not trigger this from an event an untrusted user controls, for example a comment on a fork’s pull request. The prompt is the attack surface, and the spec is part of it. Its descriptions reach the agent verbatim. That is why the agent runs in a sandbox without network, in a job without write access that runs nothing the agent wrote.

Do not give the agent job your production API key so that it can look at live responses. Run the tests against a mock, and if the agent must see real data, give it a key scoped to reads on a sandbox account.

Do not auto-merge. The guard bounds where the agent wrote and the tests bound what it broke. Neither judges whether a renamed field should be renamed for your own callers too, which is a product decision that belongs in review.

Do not use this for a spec that changes daily, or for a client a generator can rebuild. Each run costs a model call and a review, where regenerating the client costs neither.

Related how-tos

• Sandbox an agent’s shell and code execution

• Stop a coding agent inventing endpoints not in the spec

• Run an in-memory mock of your API inside SDK tests

• Install an API skill and check a coding agent uses it

• Set up a Spectral lint loop for an agent editing OpenAPI

Last verified

Verified 2026-10-15 against Node 22.22.0 and yaml 2.9.1. Every output block is what the command preceding it printed. The workflow was linted and its guard tested; it was not run on GitHub, and no command on this page called a model. Action pins were resolved with git ls-remote on the same day.