Skip to content
DashboardWebsite

LLM Agent Workflow

The client-led workflow lets an LLM agent or script review code changes through the API. It ingests a diff, claims units to analyse, submits the analyses, and fetches the report.

  1. Name the run — this is the changeset.
  2. Ingest the raw diff — the server splits it into units and returns a receipt.
  3. Drain the queue — claim a batch of units, evaluate each, and submit the batch’s analyses.
  4. Stop when a claim answers done.
  5. Fetch the report and return its link to whoever needs the result.
POST /v1/{pack}/{CHANGESET}/ingest → 201 { revision_id, unit_count, units[] }
repeat:
POST /v1/{CHANGESET}/claims {limit, revision_id, known_rulesets, known_contract}
→ status "units" { units[], rulesets{}, contract }
→ status "waiting" { retry_after_seconds }
→ status "done" → stop
<analyse each unit.body against rulesets[unit.ruleset_id], as contract says>
POST /v1/{CHANGESET}/analyses {revision_id, analyses[]} → 200 { results[], progress }
GET /v1/changesets/{CHANGESET}/report?revision={revision_id}&detail=summary&link=true

Rules travel with the claim. Each unit names its ruleset_id, and the response includes each ruleset once, along with the contract that describes the analysis payload. You do not need a separate rules-fetch call.

A changeset is the item under review, such as a pull request, pipeline, or release. Choose a stable name, such as gh-acme-billing-42 or release-4821. Each ingest into it starts a new revision unless you name the current revision. The report, merge gate, and claims use the current revision by default.

Terminal window
curl -s -X POST "http://localhost:8000/v1/python-code-change/${CHANGESET}/ingest" \
-H "Content-Type: text/plain" \
--data-binary @change.diff

A 201 response includes a receipt. Keep revision_id:

{
"changeset": "release-4821",
"revision_id": "9f3c...",
"unit_count": 12,
"pending_count": 9,
"units": [
{"unit_hash": "9c2a7e1f4d8b...", "type": "async_function", "name": "export_invoices", "action": "add", "pack": "python-code-change"}
]
}

pending_count counts the units that still need an analysis. A unit this organization already analysed with the same content keeps its analysis.

The query parameters set the rules used for every claim in this revision:

ParameterEffect
rule_typeThe pack’s rule variant, such as django. If omitted, the repository’s or pack’s default applies.
repositoryprovider/external_id, such as github/742118. Uses the repository’s rule variant and custom rules scoped to it, without rules it turned off. Packs the repository turned off do not run.
revisionAdd the units to this revision instead of starting one. It must be the changeset’s current revision.

a. Claim a batch

Terminal window
curl -s -X POST "http://localhost:8000/v1/${CHANGESET}/claims" \
-H "Content-Type: application/json" \
-d '{"limit": 5, "revision_id": "9f3c...", "known_rulesets": [], "known_contract": null}'

The response is always 200. Its status says what to do next:

statusMeaningDo
unitsunits holds 1 to limit units, now claimed by youEvaluate and submit them, then claim again.
waitingEvery remaining unit is claimed by another caller or by you in an earlier run.Claim again after retry_after_seconds, or release your own claims first.
doneEvery unit of the revision has an analysis.Fetch the report.

progress gives total, analysed, claimed, and claimable on every response.

From each unit you need:

FieldPath
Hash for the submit stepunit.unit_hash
Content to analyseunit.body
Address (type.name)unit.type + . + unit.name
Change kind (add, modify, delete)unit.action
Applicable rulesetrulesets[unit.ruleset_id].text
Rule ids a finding may namerulesets[unit.ruleset_id].rule_ids

b. Keep the rulesets and contract you receive

A ruleset is sent once per ruleset_id. Keep it, and list its id in known_rulesets on later claims; the response then leaves its text out. If a rule changes mid-run, it gets a new id and its new text comes with the next claim. GET /v1/{pack}/rules returns the same id in its ETag header.

The contract describes the analysis payload. It comes with the first batch; keep it and pass its contract_id as known_contract on later claims.

c. Evaluate each unit

Run every rule in the unit’s ruleset against unit.body to find bugs, risky changes, or breaches of your standards. For a clean unit, submit an empty findings list and say it is clean in verdict. A finding is an issue, never a note that there is none.

d. Submit the batch

Terminal window
curl -s -X POST "http://localhost:8000/v1/${CHANGESET}/analyses" \
-H "Content-Type: application/json" \
-d '{"revision_id": "9f3c...", "analyses": [{"unit_hash": "...", "verdict": "...", "findings": [...]}]}'

The response is 200 with one result per unit, in request order, and the loop’s progress:

statusWhat it meansDo
storedKept; adjustments lists what the server changed.Nothing.
rejectedNot kept; errors names the finding and field.Fix that unit and send it again.
not_in_revisionThe revision holds no such unit.Log it and move on.
claimed_by_otherAnother caller holds a live claim on the unit.Drop it.

A malformed analysis, such as one with a missing field or unknown severity, fails the whole request with 422. detail names the unit and field. A unit claimed by you or by no one is accepted.

e. Release claims for work you will not finish

A claim lasts 10 minutes or until its analysis is stored. Several agents can drain one changeset without analysing a unit twice. If you stop early, return your claims so the units are claimable again:

Terminal window
curl -s -X DELETE "http://localhost:8000/v1/${CHANGESET}/claims?revision=9f3c..."

Add &hash=<unit_hash> (repeat it) to release only some units. This does not release another caller’s claims.

{
"unit_hash": "9c2a7e1f4d8b...",
"verdict": "one to three sentences: the verdict on this unit",
"findings": [
{
"rule_id": "PYCC-001",
"title": "short finding title",
"severity": "CRITICAL | HIGH | MEDIUM | LOW | INFO",
"confidence": "HIGH | MEDIUM | LOW",
"consequence": "what breaks if this ships as it is",
"rationale": "why the rule exists: the standard, incident class or compliance clause",
"remediation": "one concrete fix, verb first",
"quotes": ["text copied verbatim from unit.body"],
"tags": ["tenant-isolation"]
}
]
}
FieldConstraint
verdictRequired and non-empty.
findingsRequired; empty for a clean unit.
rule_idOne of the ruleset’s rule_ids, or a built-in rule’s pack-file id such as PYCC-001. Use null for an issue no rule covers. Submit at most one finding per rule in a unit.
severityOne of the five levels above. Case does not matter.
confidenceOne of the three levels above. Case does not matter.
quotesOne to four verbatim substrings of unit.body, each under 200 characters. A quote not present in the body is dropped. If none remain, confidence is lowered to LOW and the finding is tagged unverified-quote.
tagsShort labels such as tenant-isolation, secrets, or logging. Stored lowercase.

The server sets the unit’s address on each finding. A finding with no rule is stored under an id derived from its title, so two different unruled issues on one unit are both kept.

Once a claim answers done, read the report with GET /v1/changesets/{CHANGESET}/report. The response contains the report itself. Add revision={revision_id} to read the revision you drained.

  • Full Markdown — ?format=markdown, the default. Includes every severity and suits output such as $GITHUB_STEP_SUMMARY.
  • Summary — ?detail=summary&link=true. Includes CRITICAL and HIGH findings in the short pull request comment layout. Return it as the agent’s final message. When report links are configured, link=true adds a link to the full HTML report.
  • JSON — ?format=json. Contains the data the report renders: each unit with its findings and triage status.
  • HTML — ?format=html. Returns a self-contained page.

An unknown parameter or value returns 422.

When report links are configured, each link=true call creates a fresh, high-entropy GET /reports/{token} link scoped to the org’s report_link_ttl_hours (24 hours by default). The setting can be changed with PATCH /v1/orgs/{org_id}/report-link-ttl; see Endpoints. Read the link from the report text (report_html_url in JSON). Do not construct or cache it. Without link=true, no link is created. The link is public and unauthenticated, but expires.

The report’s pack labels come from each unit’s own metadata.pack. You do not need to pass anything else.

When auth.api_key is enabled for the deployment, every call needs a key except the shared report link (GET /reports/{token}) and health probes (GET /health, GET /health/ready), which are always public.

Terminal window
curl -H "X-API-Key: <key>" ...

A member-scoped key covers the workflow above: rules, ingest, the analysis loop, and reports. Actions that change organization settings, such as key management under /auth/keys, LLM credentials, quota, and GitHub App wiring, need an admin or owner key.

Run more than one pack against the same changeset

Section titled “Run more than one pack against the same changeset”

Ingest each pack into one revision. The first ingest starts it; later ingests name it with revision. The loop stays the same because each unit’s rules come from the pack that produced it:

CHANGESET = "release-4821"
POST /v1/python-code-change/{CHANGESET}/ingest ← backend.diff → revision_id
POST /v1/typescript-code-change/{CHANGESET}/ingest?revision={id} ← frontend.diff
# same loop as above, unmodified
POST /v1/{CHANGESET}/claims {"revision_id": id, ...}
POST /v1/{CHANGESET}/analyses {"revision_id": id, ...}
GET /v1/changesets/{CHANGESET}/report?revision={id} # one merged report

Without revision, the second ingest starts a new revision, and the first pack’s units are no longer part of the current one. POST /v1/{CHANGESET}/ingest/auto routes a diff across every matching pack in one call and one revision.

CodeMeaningWhat to do
200OKContinue.
201IngestedContinue.
401Bad or missing API keyCheck X-API-Key.
404Not foundCheck the changeset or revision.
409revision is not current, or names another repositoryIngest without revision, or name the current one.
413Input too largeSplit it and ingest in parts.
422Payload did not validateRead detail, fix the payload, and retry.
500Server errorCheck server-side logs.
501Not implementedTry a different pack.