New provider: \<CLI name\>
Public docs: \
Public docs: <url>
Someone evaluating a CLI fills this in. Status is yes, no, or partly. Write the flag or event that proves each item, and a link to that CLI's public docs. Do not write account names, keys, or model ids that look real.
Copy the table. Leave status blank until you check the CLI.
| # | Must or nice | Status | Flag or event | Public docs |
|---|---|---|---|---|
| P1 Non-interactive mode that runs one task and exits | must | |||
| P2 Prompt from stdin, or from a file, not only argv | must | |||
| P3 Line-delimited JSON events while it works, with a final result event | must | |||
| P4 Honest exit codes, and a final result that says success or error | must | |||
| P5 Tool permission control matching read_only, edit, and interview, without skipping checks | must | |||
| P6 Model selection by flag | must | |||
| P7 Auth from its own login or an environment variable, with a clear "not signed in" error | must | |||
| P8 Usage in the final result (input, output, cache tokens); per-call usage for context size | nice | |||
| P9 A session id, and resume by id | nice | |||
| P10 Control of which MCP servers and skills load, and a way to list them | nice | |||
| P11 A way to turn off its own sandbox when the factory sandbox wraps it, and documented state folders | nice | |||
| P12 --help and --version that Probe can read without a model call | must | |||
| P13 Ends cleanly on SIGTERM (Unix) or Job Object close (Windows) | must | |||
| P14 Doesn't send the prompt to a cloud agent or remote worker by default | must |
Verdict
- Every must item is
yes→ can be a built-in provider. Add a runner package, a fake, a flags file, and pass the conformance suite. - Any must item is
no(orpartly) → use a custom command step. See Using a CLI that isn't a built-in provider.
Filled examples for Claude Code and Cursor are in the checklist appendix.