CLI Reference

Entry point: securebench (module securebench.cli:main).

`securebench run`

Run a tester YAML benchmark-pack harness.


securebench run --config PATH [options]

Arguments

Argument	Required	Description
`--config PATH`	yes	Path to tester YAML file

Options

Option	Default	Description
`--env-file PATH`	`.env`	Dotenv file loaded before run (`securebench.env.load_env_file`)
`--limit N`	none	Process first N compiled tasks only
`--output-dir PATH`	from YAML	Override `run.output_dir`
`--resume`	off	Keep valid lines in existing `candidates.jsonl`; skip completed `task_id`
`--quiet`	off	Use `NullProgressReporter`
`--show-command-output`	off	Include sandbox stdout/stderr snippets in progress
`--show-agent-output`	off	Stream Codex agent messages; write `agent-trace.log`

Exit codes

Code	Meaning
`0`	Run completed; summary printed to stdout
`1`	Error (`ConfigError`, `ImportError`, `OSError`, `ValueError`) — message on stdout
`2`	Unknown subcommand (argparse)

Success output


securebench: run_id=<id> total=<n> verification=<status> verified=<n> passed=<n> output=<path>

verification is complete, partial, or pending (aggregate — see Architecture).

Module invocation

Equivalent to the installed script:


python -m securebench.cli run --config PATH

Environment loading

load_env_file() parses KEY=VALUE lines:

Supports optional export prefix
Strips single/double quotes from values
Does not override existing environment variables unless override=True (CLI uses default override=False)
Missing env file is silently ignored (returns {})

Python equivalent


from securebench.env import load_env_file
from securebench.tester_config import load_tester_config
from securebench.tester_run import run_tester_config, with_tester_overrides
 
load_env_file(".env")
config = load_tester_config("path/to/tester.yaml")
config = with_tester_overrides(config, output_dir="runs/custom")
summary = run_tester_config(config, limit=1, resume=False)