# How to run a coding agent headless in CI to keep a client current > Hand an upstream spec change to an agent in a read-only CI job, guard its patch, and open the pull request from a job that never holds the model key. Source: https://voxgig.com/howto/run-a-coding-agent-headless-in-ci - Audience: platform-team - Level: advanced - Verified: 2026-10-15 - Published: 2026-10-15 ## Short answer Split the workflow by what each job holds. The agent job holds the model key and only contents: read, checks out with persist-credentials: false, runs the agent with a turn limit and a sandbox, and exports the diff. A job without the key guards that patch and runs the tests against a mock. A third, with write access and no key, opens the pull request with a GitHub App token, so required checks gate the merge. --- ## You will need A GitHub repository with Actions enabled, and a model API key for one agent. A GitHub App that can push branches and open pull requests, and tests that run against a mock rather than the live API. Verified 2026-10-15 against Node 22.22.0 and `yaml` 2.9.1. The prompt, and the diff of the spec it carries, are a separate task, and so is building the mock. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [Claude Code GitHub Action (`claude -p`)](https://code.claude.com/docs/en/github-actions) | You want the prompt, the tool allowlist, and the sandbox settings in your own workflow file | Acts as the Claude GitHub App, with that App's write access, unless you pass the job's token | You cannot install bubblewrap on the runner, which leaves its sandbox off | | [Codex CLI `codex exec`](https://learn.chatgpt.com/docs/non-interactive-mode) | You want a read-only sandbox by default and an action that keeps the key behind a proxy | `codex exec` writes only when a run opts in, but the action defaults to a writable workspace and removes `sudo` for the rest of the job | Later steps in the same job need `sudo` | | [Gemini CLI in non-interactive mode](https://geminicli.com/docs/cli/headless/) | Gemini models, with tool rules in policy files and a container sandbox | The sandbox is a Docker container, so the runner needs a container runtime and an image | You want the sandbox without a container runtime | | [GitHub Copilot coding agent](https://docs.github.com/en/copilot/concepts/security-governance-and-network-settings/risks-and-mitigations) | You want GitHub to run the environment, the firewall, and the branch rules for you | Less control over tools and prompt, and by default its pull requests wait for a person to approve their workflow runs | You need the run to finish with no person in the loop | The three CLI routes put the prompt, the sandbox flags, and the permissions in a file you review, and leave you to contain them. Copilot's agent arrives contained: it pushes only to a `copilot/` branch, works behind a firewall, and by default its pull requests wait for a person before their workflows run. That containment is the cost as well, because you choose fewer of its tools and its runs stop at that approval unless you switch it off. ## Keep the model key out of the job that can write The split comes from the Codex documentation, which gives the agent job [`contents: read` only](https://learn.chatgpt.com/docs/non-interactive-mode) and a second job the write permissions without the key. It applies to every route. A `detect` job fetches the published spec on a weekday schedule[^1] and stops the run when it matches the vendored copy. The agent job starts like this: ```yaml title="workflow/spec-sync.yml" agent: needs: detect if: needs.detect.outputs.changed == 'true' runs-on: ubuntu-latest timeout-minutes: 30 permissions: contents: read steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false ``` [`persist-credentials: false`](https://github.com/actions/checkout) stops checkout leaving the job token where every later `git` command, the agent's included, can use it. The agent step then fixes the turn limit, the permission mode, and the tools: ```yaml title="workflow/spec-sync.yml" - name: Run the agent uses: anthropics/claude-code-action@374f254f5efaf968abfe15d399fd90972e8138b9 # v1.0.251 with: anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} github_token: ${{ github.token }} settings: ci/claude-settings.json prompt: | The upstream OpenAPI document changed. spec/openapi.json is the new version and spec-diff.txt shows what changed. Update the client in src/ so that `npm test` passes against the mock in test/. Change files under src/ and test/ only. Do not skip, delete or weaken a test. If the change cannot be made without breaking a caller, stop and explain why in your final message. claude_args: >- --max-turns 40 --permission-mode dontAsk --allowedTools "Read,Edit,Write,Bash(npm test),Bash(node --test *)" --disallowedTools "WebFetch,WebSearch" # The agent's edits are code, ci/guard.mjs included. Running the guard or the tests here # would run that code outside the sandbox, in the job that holds the key, so this job # only exports the diff. - name: Export the change as a patch run: | rm -f spec-diff.txt git add -N . git diff --binary HEAD > "$RUNNER_TEMP/agent.patch" ``` `github_token` is the line that matters most. Without it the action [authenticates as the Claude GitHub App](https://code.claude.com/docs/en/github-actions), whose permissions do not depend on the job's. `dontAsk` [denies every call that would prompt](https://code.claude.com/docs/en/headless), so the agent gets the tools listed and nothing else. `ci/claude-settings.json` turns the [sandbox](https://code.claude.com/docs/en/sandboxing) on with an empty domain allowlist and `failIfUnavailable`. Under `dontAsk` the sandbox refuses a shell command's connection to any host off the list, and a runner that cannot start the sandbox stops the run. The tests still reach their mock, because `test/mock.mjs` starts it inside the test command. On Linux each command in the sandbox gets a `localhost` private to it, so a mock started by an earlier step would be out of reach. The job stops at the diff, because everything the agent wrote, `ci/guard.mjs` included, is code a later step would run outside the sandbox beside the key. The [Codex example](https://learn.chatgpt.com/docs/non-interactive-mode) also only serializes the diff after the agent. Its page warns that tests can read a key set in their job's environment. A `verify` job without the key guards the patch, applies it, and runs `npm test`. The other two CLIs fit the same slot. Codex takes the sandbox as an input: ```yaml - name: Run the agent uses: openai/codex-action@bdf19a4a223ec2549a3e2274a0cf61556bc07675 # v1.13 with: openai-api-key: ${{ secrets.OPENAI_API_KEY }} prompt-file: ci/spec-sync-prompt.md permission-profile: ":workspace" safety-strategy: drop-sudo ``` Gemini CLI takes it as settings. Its [action passes `--yolo`](https://github.com/google-github-actions/run-gemini-cli/blob/v0.1.22/action.yml) on every run, which approves every tool call whatever `defaultApprovalMode` says. The [configuration reference](https://geminicli.com/docs/reference/configuration/) describes `security.disableYoloMode` as disabling YOLO mode even when a flag enables it: ```yaml - name: Run the agent uses: google-github-actions/run-gemini-cli@f77273f4c914e4bf38440cf36a0369cb64a37489 # v0.1.22 with: gemini_api_key: ${{ secrets.GEMINI_API_KEY }} prompt: Update src/ for the spec change in spec/openapi.json. Change src/ and test/ only. settings: | { "general": { "defaultApprovalMode": "auto_edit" }, "security": { "disableYoloMode": true }, "tools": { "sandbox": "docker", "sandboxNetworkAccess": false } } ``` Copilot has no step to put there. The workflow opens an issue and assigns it to `copilot-swe-agent[bot]` through the [issues API](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/cloud-agent/use-cloud-agent-via-the-api), and the agent runs in its own environment. The documentation describes that call with a personal access token, not the job's token: ```bash gh api --method POST "/repos/$GITHUB_REPOSITORY/issues" --input - <<'JSON' { "title": "Update the client for the upstream spec change", "body": "spec/openapi.json changed upstream. Update src/ so npm test passes. Change src/ and test/ only.", "assignees": ["copilot-swe-agent[bot]"] } JSON ``` ## Guard the patch, not the agent's account of it The guard reads the diff and refuses anything outside `src/` and `test/`, a deleted test, a skipped test, a symbolic link, a binary file, or a change over 400 lines. It reads the patch the way `git apply` will, which takes a plain `--- a/package.json` header anywhere outside a hunk. So the guard counts each hunk's lines from its header and refuses any line it does not expect: ```js title="ci/guard.mjs" if (oldLeft > 0 || newLeft > 0) { if (line.startsWith('+') && newLeft > 0) { cur.added.push(line.slice(1)); newLeft-- } else if (line.startsWith('-') && oldLeft > 0) { cur.removed++; oldLeft-- } else if ((line.startsWith(' ') || line === '') && oldLeft > 0 && newLeft > 0) { oldLeft--; newLeft-- } else if (line !== '\\ No newline at end of file') fail(n, line) continue } ``` Each finding then comes from a file record: ```js title="ci/guard.mjs" for (const f of files) { if (f.unparsed) { findings.push(`cannot read this patch at ${f.from}`); continue } // A rename has two paths and both have to be in bounds: moving a test out of test/ deletes it. for (const path of new Set([f.from, f.to])) { if (!ALLOWED.some((re) => re.test(path))) findings.push(`${path}: outside src/ and test/`) } if (f.binary) findings.push(`${f.to}: binary change`) if (f.symlink) findings.push(`${f.to}: adds a symbolic link`) if (/^test\//.test(f.from) && (f.status === 'deleted' || (f.status === 'renamed' && !/^test\//.test(f.to)))) { findings.push(`${f.from}: deletes a test file`) } if (/^test\//.test(f.to)) { for (const added of f.added) if (SKIPS.test(added)) findings.push(`${f.to}: skips a test: ${added.trim()}`) } changed += f.added.length + f.removed } ``` `pipeline.mjs` runs the agent and verify steps locally against a copy of `project/`, a client written for spec 1.4.0. It swaps in spec 2.0.0, where `kwh` became a decimal `value` and a `unit`, and then lets the agent edit twice. The agent is `stub-agent.mjs`, a script with two scripted edits, so no model is called. ```bash node pipeline.mjs ``` ```text output spec 1.4.0, tests: pass 2, fail 0, skipped 0 spec 2.0.0, tests: pass 1, fail 1, skipped 0 run 1 agent (fix) changed: src/client.mjs guard: ok, 7 changed lines tests after the agent: pass 2, fail 0, skipped 0 run 2 agent (cheat) changed: .github/workflows/ci.yml, package.json, test/client.test.mjs guard: refused, 3 findings .github/workflows/ci.yml: outside src/ and test/ package.json: outside src/ and test/ test/client.test.mjs: skips a test: test.skip('sums a meter\'s readings in kWh', async () => { tests after the agent: pass 1, fail 0, skipped 1 ``` Read the last line of run 2. The tests pass, with one skipped, so a gate that runs only the tests would let it through. The guard refuses it on all three edits. ## Lint the workflow so the split survives edits GitHub runs a workflow that breaks every rule on this page without complaint, so `lint-workflow.mjs` checks the file itself. It wants top-level `permissions: {}` and no write permission in a job that holds a model key. It also wants no key in `env`, credentials not persisted, actions pinned to a commit, a timeout, and a turn limit. After the agent step, that job may run nothing from the checkout, and a job without the key has to run the guard. Run it on the workflow and on a one-job first draft: ```bash node lint-workflow.mjs workflow/spec-sync.yml fixtures/one-job.yml ``` ```text output ok workflow/spec-sync.yml FAIL fixtures/one-job.yml workflow: no top-level permissions, so a job without its own block gets the repository default workflow: no concurrency group, so two spec changes can race to open two pull requests jobs.sync: a model key in job env reaches every step, npm install scripts included jobs.sync: actions/checkout@v7 is not pinned to a commit jobs.sync: anthropics/claude-code-action@v1 is not pinned to a commit jobs.sync: holds a model key and a write permission in the same job jobs.sync: runs an agent with no timeout-minutes jobs.sync: checkout persists the job token, so every git command the agent runs is authenticated jobs.sync: the agent runs with permission checks or the sandbox turned off jobs.sync: claude_args sets no --max-turns jobs.sync: no github_token, so the action acts as the Claude GitHub App with its own write access workflow: no job without the model key guards the patch ``` The linter finds an agent job by its secret, so it knows the key names it was written for. Add yours to `MODEL_KEY` if it is called something else. For the Gemini action it also wants `security.disableYoloMode`, because the action's `--yolo` never appears in the workflow file. ## Check it worked The suite builds each patch with real `git` and checks the linter against fifteen edits to the workflow, each of which must produce exactly one finding. It also holds four hand-written patches that `git apply --check` accepts and the guard refuses: ```bash npm test ``` ```text output # tests 33 # suites 0 # pass 33 # fail 0 ``` The end-to-end check needs GitHub and a model key, so this page did not run it. In a test repository, point the detect step's URL at a copy of the spec with one field renamed, then run the workflow and read the result: ```bash gh workflow run spec-sync.yml RUN=$(gh run list --workflow spec-sync.yml --limit 1 --json databaseId --jq '.[0].databaseId') gh run watch "$RUN" --exit-status gh pr diff "agent/spec-sync-$RUN" --name-only gh pr checks "agent/spec-sync-$RUN" --required --watch ``` `gh pr diff` should list `spec/openapi.json` and files under `src/` or `test/` only, and `gh pr checks` should end with every required check passing. ## When it goes wrong The pull request has no checks, only a banner asking someone to approve its runs. It was opened with `GITHUB_TOKEN`, and a pull request opened with that token [creates its runs in an approval-required state](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow).[^2] Open it with a GitHub App token, as `open-pr` does. The agent's shell commands reached the network although the settings turned the sandbox on. Without `failIfUnavailable`, Claude Code runs commands outside the sandbox when it cannot start, as on a runner without `bubblewrap` and `socat`. Install both, keep `failIfUnavailable` and `allowUnsandboxedCommands: false`, and add the AppArmor profile, because on Ubuntu 24.04 the default policy stops bubblewrap creating the user namespaces it needs. A pinned SHA does not match the commit the release points to. For an annotated tag, `git ls-remote` prints the tag object first and the commit on a second line ending `^{}`, and GitHub's [hardening guide](https://docs.github.com/en/actions/reference/security/secure-use) asks for the commit. Two of the pins on this page are annotated tags. ## When not to do this Do not trigger this from an event an untrusted user controls, for example a comment on a fork's pull request. The prompt is the attack surface, and the spec is part of it. Its descriptions reach the agent verbatim. That is why the agent runs in a sandbox without network, in a job without write access that runs nothing the agent wrote. Do not give the agent job your production API key so that it can look at live responses. Run the tests against a mock, and if the agent must see real data, give it a key scoped to reads on a sandbox account. Do not auto-merge. The guard bounds where the agent wrote and the tests bound what it broke. Neither judges whether a renamed field should be renamed for your own callers too, which is a product decision that belongs in review. Do not use this for a spec that changes daily, or for a client a generator can rebuild. Each run costs a model call and a review, where regenerating the client costs neither. ## Related how-tos - [Sandbox an agent's shell and code execution](/howto/sandbox-agent-shell-and-code-execution) - [Stop a coding agent inventing endpoints not in the spec](/howto/stop-a-coding-agent-inventing-endpoints) - [Run an in-memory mock of your API inside SDK tests](/howto/run-an-in-memory-mock-inside-sdk-tests) - [Install an API skill and check a coding agent uses it](/howto/install-an-api-skill-for-a-coding-agent) - [Set up a Spectral lint loop for an agent editing OpenAPI](/howto/spectral-lint-loop-for-a-coding-agent) ## Last verified Verified 2026-10-15 against Node 22.22.0 and `yaml` 2.9.1. Every output block is what the command preceding it printed. The workflow was linted and its guard tested; it was not run on GitHub, and no command on this page called a model. Action pins were resolved with `git ls-remote` on the same day. [^1]: The schedule in `workflow/spec-sync.yml` fires at 06:17, not 06:00. GitHub's [events reference](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows) names the start of every hour as a time of high load, when scheduled runs can be delayed and, under enough load, some queued jobs dropped. It advises another minute of the hour. It does not say which minute is quiet, only which one is not. [^2]: GitHub's reason for withholding runs from `GITHUB_TOKEN` events is recursion: a workflow that pushed with the token would otherwise trigger itself, and then again. Pull requests got an exception, runs that wait for a person with write access, so that a pull request opened by automation can be tested at all. A [GitHub App token](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow) is one documented way round the wait, and a personal access token is the other.