CLI reference

Entry point: securebench (securebench.cli:main).

`securebench run`

Run a tester YAML benchmark-pack harness.


securebench run --config PATH [options]

Arguments

Argument	Required	Description
`--config PATH`	yes	Tester YAML path

Options

Option	Default	Description
`--env-file PATH`	`.env`	Dotenv file loaded before the run
`--limit N`	none	Process the first `N` compiled tasks
`--output-dir PATH`	from YAML	Override `run.output_dir`
`--resume`	off	Keep valid lines in existing `candidates.jsonl`; skip completed `task_id`
`--quiet`	off	Use `NullProgressReporter`
`--show-command-output`	off	Include sandbox stdout/stderr snippets in progress output
`--show-agent-output`	off	Stream Codex agent messages and write `agent-trace.log`

Run exit codes

Code	Meaning
`0`	Run completed; summary printed to stdout
`1`	Handled error such as `ConfigError`, `ImportError`, `OSError`, or `ValueError`; message printed to stdout
`2`	Unknown subcommand or argument parsing error

Success output


securebench: run_id=<id> total=<n> verification=<status> verified=<n> passed=<n> output=<path>

verification is aggregate status: complete, partial, or pending. See Architecture.

Module invocation


python -m securebench.cli run --config PATH

`securebench audit`

Audit a tester YAML benchmark-pack config and write a JSON report.


securebench audit --config PATH --output-dir DIR [--limit N]

The audit checks compiled tasks for visibility mistakes, unsafe agent materialization, result redaction, repo-patch default policy behavior, missing repo-patch implementation allowlists, and benchmark manifest provenance.

The command exits 1 when any finding has failed status.

`securebench audit-self`

Run built-in SecureBench security audits.


securebench audit-self --output-dir DIR [--static-only] [--skip-docker]

--static-only skips dynamic smoke checks. --skip-docker records Docker-dependent checks as skipped.

Audit exit codes

Code	Meaning
`0`	Audit completed without failed findings
`1`	Audit failed, or command raised a handled error
`2`	Argument parsing error

Environment loading

load_env_file() parses KEY=VALUE lines:

supports optional export prefix
strips single or double quotes from values
does not override existing environment variables unless override=True
silently ignores a missing env file

The CLI uses override=False.

Python equivalent


from securebench.env import load_env_file
from securebench.tester_config import load_tester_config
from securebench.tester_run import run_tester_config, with_tester_overrides
 
load_env_file(".env")
config = load_tester_config("path/to/tester.yaml")
config = with_tester_overrides(config, output_dir="runs/custom")
summary = run_tester_config(config, limit=1, resume=False)