Review Feedback
Just want to do it in the dashboard? See the guide: Dismiss a false positive or Find the rules that misfire.
This page describes the feedback settings for platform admins. These settings control whether reviewer feedback reaches analysis prompts, how much feedback is included, and when the system proposes rule changes. For the feedback process, see the review feedback loop.
In the dashboard: Platform Admin → Analysis → Review Feedback. A platform admin can update these settings with PATCH /v1/admin/settings. Changes apply to the next analysis without a restart.
Recording reviewer decisions has no switch. The settings below control how the system uses those decisions. Calibration adds feedback to analysis prompts, while proposal thresholds can suggest rule changes for a human to review.
Configure calibration
Section titled “Configure calibration”| Field | Default | Bounds | What it does |
|---|---|---|---|
calibration_enabled | true | — | Whether reviewer feedback is included in analysis prompts. When off, feedback is still recorded, shown in Rule Health, and used to generate proposals. |
min_samples | 3 | 1–100 | Minimum number of decisive verdicts (confirmed and rejected) before a rule can calibrate. unclear verdicts do not count. |
max_entries_per_rule | 5 | 1–50 | Maximum reviewer comments quoted per rule, newest first. |
max_chars | 4000 | 200–40000 | Maximum size of the full calibration block. The block accompanies every unit in every run. If it exceeds this limit, the system drops whole rule entries from the end. |
severity_ceiling | HIGH | CRITICAL, HIGH, MEDIUM, LOW | Highest severity calibration can affect. Findings above this severity are always reported at full volume. The default HIGH leaves CRITICAL findings unaffected. Setting this to CRITICAL means no severity is exempt. |
trusted_only | true | — | Whether only trusted feedback can reach an analysis prompt. See the warning below. |
min_samples avoids calibrating from very few verdicts. A new organization may have no calibration for weeks. Rule Health shows why a rule is not calibrating.
Configure comment ingestion
Section titled “Configure comment ingestion”| Field | Default | Bounds | What it does |
|---|---|---|---|
pr_comments_enabled | true | — | Whether pull-request comments and merge-request notes are ingested as feedback. This setting is independent of calibration. You can collect comments without including them in prompts, or calibrate from dashboard triage without ingesting comments. |
command_prefix | @opentremor | 2–64 characters | Prefix required for an explicit verdict in a pull-request comment. |
Set rule proposal thresholds
Section titled “Set rule proposal thresholds”These thresholds use decisive verdicts. When a threshold is met, the system creates a proposal for a person to review. It does not change the rule automatically.
| Field | Default | Bounds | Fires when |
|---|---|---|---|
proposal_rejection_threshold | 0.6 | 0.0–1.0 | The rejection rate reaches this value. The system proposes lowering the rule’s severity by one level, when possible. For a rule already at LOW, it can instead propose requiring review. |
proposal_disable_threshold | 0.9 | 0.0–1.0 | The rejection rate reaches this value. The system proposes disabling the rule. |
proposal_confirm_threshold | 0.9 | 0.0–1.0 | The confirmation rate reaches this value for a requires_review rule. The system proposes clearing the review requirement. |
Example
Section titled “Example”{ "feedback_calibration_enabled": true, "feedback_min_samples": 5, "feedback_severity_ceiling": "MEDIUM", "feedback_max_chars": 2000}This configuration requires five decisive verdicts before calibration. It also leaves findings with severity HIGH or CRITICAL unaffected by feedback.
Retention
Section titled “Retention”Feedback is kept for the period set by retention.feedback_days in the retention section. The retention setting controls when feedback is deleted, through the retention sweep and its POST /v1/admin/retention/run endpoint.