Log store connectors

A log store connector is a connector with a logs block. It adds recent log lines to a run as fenced context, and turns critical error groups into held factory tasks. Items are factory tasks, not tracker tickets.

MappingCopy Anchor Link

connectors:
  demo-logs:
    mcp:
      url: "https://logs.example.invalid/mcp"
      headers:
        Authorization: { secret: true }
    context:
      recent: { tool: search_logs, args: { query: "service:{{task.labels.service}} status:error", from: "now-1h" } }
    agent_tools: [search_logs]
    logs:
      errors:
        tool: search_error_groups
        args: { query: "status:error", from: "now-15m" }
        every: 5m
        event: { id: id, group: issue_id, message: message, service: service, level: status, kind: error.type, stack: error.stack, count: count, first_seen: first_seen, last_seen: last_seen, url: url }
      rules:
        - when: { level: [critical, fatal], min_count: 1 }
          severity: critical
        - when: { level: [error], min_count: 20, window: 15m }
          severity: critical
        - when: { level: [error] }
          severity: low
      create_for: [critical]
      items: hold
      reopen_after: 24h
      repos: [demo]

logs.errors is a read use. The broker calls it every every (1 minute to 24 hours, default 5 minutes) while serve runs, with the same backoff and pinning as intake. event maps fields by dotted path: id and message are required.

A log store connector has no tracker block and needs no write-back. It may still declare write-back tools; those follow (effects, outside, held at the gate).

logs.items: hold is the default, including when the connector's intake uses mode: queue. Only logs.items: queue queues immediately. At most 5 new items per poll and 20 per day per connector; further fingerprints stay seen with a severity.

FingerprintsCopy Anchor Link

When the store gives a grouping id (event.group), the fingerprint is ext: plus that id. Otherwise the factory hashes service, kind, and a normalised message and stack (SHA-256, first 16 hex characters). Normalisation turns numbers, hex strings of 8 or more characters, UUIDs, quoted strings, URLs, temp paths and timestamps into placeholders, and keeps the first 5 stack frames without line numbers or memory addresses.

One fingerprint is one work item at most. New events update last_seen and count. A running run for that fingerprint gets a run note (seen 40 more times since this run started) and nothing else.

When the fingerprint's task is done and the error is seen again more than reopen_after (default 24 hours) after the task ended, a new held item is created with the label regression and a link to the earlier task.

No item is created when the fingerprint already has a task, is ignored, or is linked to a ticket. kusaw logs link <connector>:<fingerprint> <tracker-connector>:<id> links by hand. If a tracker connector has context.search, a hit is shown on the held item as a possible ticket and is not linked automatically.

RedactionCopy Anchor Link

Log context is redacted first, then cut, then fenced as connector:<name>:

• secret-looking values
• bearer tokens (Bearer …)
• keys in URLs (token=, api_key=, secret=, and similar)
• email addresses

Cuts: at most 200 lines or 4 KiB per event, 100 KiB per file, newest first. The file starts with the query, the time window, and counts (showing 200 of 1,834 lines). A canary secret in a log line never reaches a prompt, an event, or a serve log.

Agent tools for log search are refused for agents unless the server marks them read-only or the entry says allow_unmarked: true. Results are fenced and redacted the same way.

CLI and APICopy Anchor Link

Held log items show under kusaw connector items --held and start with kusaw connector accept. Fingerprints:

kusaw logs fingerprints [connector] [--severity <s>]
kusaw logs ignore <connector>:<fingerprint>
kusaw logs link <connector>:<fingerprint> <tracker-connector>:<id>

API: GET /api/f/{factory}/connectors/{name}/fingerprints. There is no Dashboard screen.

Events: logs.fingerprint_new {connector, fingerprint, severity}, logs.item_held, connector.limit_reached. No log text, secret, or item body.

A run whose task came from a log-store item gets context.logs and env KUSAW_LOG_FINGERPRINT in every workflow.

DatadogCopy Anchor Link

See examples/connectors/datadog/kusaw.yaml. It uses Datadog's official MCP server over hosted streamable HTTP (for example https://mcp.datadoghq.com/v1/mcp), so kusaw validate warns that it is an outside service. Auth is a header secret: Authorization: Bearer with a personal or service access token, or DD_API_KEY and DD_APPLICATION_KEY. Use a token or application key with read scope only.

logs.errors uses search_datadog_error_tracking_issues (toolset error-tracking); the issue id is the group. Context and agent tools use search_datadog_logs (toolset core). Tool names come from Datadog's public tools page. kusaw connector check pins what the server lists. If a tool is not marked read-only, leave allow_unmarked off.