Skip to content

runbooks / command

Runbook to test script

Writes or adopts the test script of a runbook, in the project's test library

❯ /runbooks:to-test <id> [--out path] [--model <model>]

When executing as a child, you cannot ask the user. When the main session executes this file inline, it asks the user directly and handles calls through the dispatcher. Where this file says to ask (AskUserQuestion, "ask the user", a diff approval), stop with status: question as agents.md says. Where it says to say something and stop, put the message in summary: and report status: done. Commit nothing: the main session commits after it prints your summary.

The runbook is the source of truth. A script changes to match the runbook; the runbook changes only when the user says it is the one that is wrong, and then through /runbooks:update.

run also reaches this file for an api runbook whose script is missing or out of sync, or whose failure looks like a script mistake. Two cases. When run launched you for it (escalation), your brief says what to skip. When a run sub-agent follows this file itself (Forced model: yes), skip item 6 (the run is that run) and item 9. Do not end with a report: list the script in the run's files: and go on with the run. In both cases the run commits the script, so item 9 only lists the script in files: and names its path and the scripts: change in summary:.

With role tracking on (roles.enabled in the config), also read the Roles part of "Scripts of api runbooks" in format.md, and for a runbook with cases the "Contexts and cases" section. The role rules below apply only then, and the case rules only to a runbook with cases. A script written before tracking was on has no role in its test names, so it no longer matches its runbook. Rewrite it as item 5 says and show the renamed tests in the mapping of item 3.

Parameters

Parameter What it does
<id> The runbook to script. A placeholder has no steps and is refused. When the project already has tests of the same flow, the command offers to adopt that file instead of writing a new one.
--out <path> Where to write a new script. Defaults to <tests.<surface>.dir>/<capability>/<flow>.runbook.<ext>, under tests.<surface>.root when it is set, with the extension and naming of the project's tests (.spec.ts, .cy.ts, .test.ts, Test.php). With a UI tests repository, the script goes there and gets a commit of its own in that repository. See UI tests in another repository.
--model <m> Run this invocation on one model: inline (in the main session), haiku, sonnet, opus or fable. Any value but inline runs the command in a sub-agent on that model. Without it, the command takes its row of the models table in the config: by default to-test runs on opus. On Codex, the values are inline, inherit (a sub-agent on the session's model) or a model ID the session supports, and the table is codex.models, where every row that is not inline defaults to inherit. See models.
/runbooks:to-test billing.issue-invoice                                                      # write or adopt, in tests.<surface>.dir
/runbooks:to-test billing.create-invoice --out tests/Feature/Runbooks/CreateInvoiceTest.php  # to a path you choose
/runbooks:to-test billing.issue-invoice --model sonnet                                       # on sonnet, this time only

View source on GitHub