Skip to content
DashboardWebsite

Review Feedback

Just want to do it in the dashboard? See the guide: Dismiss a false positive or Find the rules that misfire.

This page describes the feedback settings for platform admins. These settings control whether reviewer feedback reaches analysis prompts, how much feedback is included, and when the system proposes rule changes. For the feedback process, see the review feedback loop.

In the dashboard: Platform Admin → Analysis → Review Feedback. A platform admin can update these settings with PATCH /v1/admin/settings. Changes apply to the next analysis without a restart.

Recording reviewer decisions has no switch. The settings below control how the system uses those decisions. Calibration adds feedback to analysis prompts, while proposal thresholds can suggest rule changes for a human to review.

FieldDefaultBoundsWhat it does
calibration_enabledtrue—Whether reviewer feedback is included in analysis prompts. When off, feedback is still recorded, shown in Rule Health, and used to generate proposals.
min_samples31–100Minimum number of decisive verdicts (confirmed and rejected) before a rule can calibrate. unclear verdicts do not count.
max_entries_per_rule51–50Maximum reviewer comments quoted per rule, newest first.
max_chars4000200–40000Maximum size of the full calibration block. The block accompanies every unit in every run. If it exceeds this limit, the system drops whole rule entries from the end.
severity_ceilingHIGHCRITICAL, HIGH, MEDIUM, LOWHighest severity calibration can affect. Findings above this severity are always reported at full volume. The default HIGH leaves CRITICAL findings unaffected. Setting this to CRITICAL means no severity is exempt.
trusted_onlytrue—Whether only trusted feedback can reach an analysis prompt. See the warning below.

min_samples avoids calibrating from very few verdicts. A new organization may have no calibration for weeks. Rule Health shows why a rule is not calibrating.

FieldDefaultBoundsWhat it does
pr_comments_enabledtrue—Whether pull-request comments and merge-request notes are ingested as feedback. This setting is independent of calibration. You can collect comments without including them in prompts, or calibrate from dashboard triage without ingesting comments.
command_prefix@opentremor2–64 charactersPrefix required for an explicit verdict in a pull-request comment.

These thresholds use decisive verdicts. When a threshold is met, the system creates a proposal for a person to review. It does not change the rule automatically.

FieldDefaultBoundsFires when
proposal_rejection_threshold0.60.0–1.0The rejection rate reaches this value. The system proposes lowering the rule’s severity by one level, when possible. For a rule already at LOW, it can instead propose requiring review.
proposal_disable_threshold0.90.0–1.0The rejection rate reaches this value. The system proposes disabling the rule.
proposal_confirm_threshold0.90.0–1.0The confirmation rate reaches this value for a requires_review rule. The system proposes clearing the review requirement.
{
"feedback_calibration_enabled": true,
"feedback_min_samples": 5,
"feedback_severity_ceiling": "MEDIUM",
"feedback_max_chars": 2000
}

This configuration requires five decisive verdicts before calibration. It also leaves findings with severity HIGH or CRITICAL unaffected by feedback.

Feedback is kept for the period set by retention.feedback_days in the retention section. The retention setting controls when feedback is deleted, through the retention sweep and its POST /v1/admin/retention/run endpoint.