Progress Events

SecureBench emits progress events through securebench.progress.emit_progress(). The CLI uses StreamProgressReporter unless --quiet.

Events are formatted as human-readable lines on stderr. Structured fields vary by event.

Event catalog

Event	Typical fields	When
`run_start`	`run_id`, `output_dir`	Run begins
`resume`	`completed`, `remaining`, `output_path`	`--resume` skipped tasks
`task_start`	`index`, `total`, `task_id`, `family`	Before producer
`producer_start`	`task_id`	Harness starting
`producer_done`	`task_id`, `candidate_kind`, `status`, `failure_reason`	After extraction or timeout
`verifier_start`	`task_id`	Verifier starting
`verifier_done`	`task_id`, `status`, `score`, `phase`	Verifier finished
`task_done`	`task_id`, `status`, `passed`, `score`	Task complete
`sandbox_create`	`kind`, `image`, `root`, `network`, `read_only`	Sandbox constructed
`sandbox_command`	`kind`, `image`, `workdir`, `command`	Before command
`sandbox_result`	`kind`, `exit_code`, `command`, `stdout`, `stderr`	After command
`agent_output`	Codex-specific	With `--show-agent-output`

CLI flags

Flag	Effect
`--quiet`	No progress output
`--show-command-output`	Include stdout/stderr snippets on `sandbox_result`
`--show-agent-output`	Stream agent messages; reset `agent-trace.log` each run

Agent trace file

When --show-agent-output is set, events named agent_output append to:


<output_dir>/agent-trace.log

The file is truncated at each run_start.

Custom reporter


from securebench.progress import ProgressReporter, progress_context
 
class MyReporter(ProgressReporter):
    def event(self, name, **fields):
        print(name, fields)
 
with progress_context(MyReporter()):
    run_tester_config(config)

Or pass progress= directly to run_tester_config().