MongoDB Backend

Collections

Every collection is scoped to an org_id, with one exception (allowed_models, platform-wide) — see Architecture — Data model for the full picture. The two most frequently touched at request time:

resources

One document per unique (org, content hash) pair.

{
  "_id":      "acme-corp:a3f1c2d4e5b6…",
  "org_id":   "acme-corp",
  "hash":     "a3f1c2d4e5b6…",
  "value": {
    "type":     "aws_db_instance",
    "name":     "orders",
    "action":   "create",
    "body":     "resource \"aws_db_instance\" …",
    "metadata": {"analyzer": "terraform-plan"}
  },
  "analysis": null,
  "last_ingested_at": "2026-07-15T08:59:40.890869+00:00"
}

_id is "{org_id}:{hash}" — MongoDB’s native unique constraint enforces per-org deduplication (not global: the same content submitted by two orgs is stored twice, deliberately, since value.body can carry tenant-identifying data). analysis starts as null and is updated once an analysis (client- or server-led) is submitted. last_ingested_at is stamped on creation and refreshed on every add_to_namespace call (including a re-ingest of the same hash) — the age signal Data Retention’s sweep uses to redact value (not the whole document — analysis/hash survive) once it’s stale.

namespace_entries

One document per (org, namespace, resource) triple.

{
  "_id":       "acme-corp:production:a3f1c2d4e5b6…",
  "org_id":    "acme-corp",
  "namespace": "production",
  "hash":      "a3f1c2d4e5b6…",
  "created_at": "2026-07-15T08:59:40.890869+00:00"
}

_id is "{org_id}:{namespace}:{hash}" — guarantees idempotent upserts with $setOnInsert, which is also why created_at is only ever set on the first insert for a given (org, namespace) pair ($setOnInsert, not $set) — it’s the namespace’s creation date, not the entry’s. GET /namespaces groups this collection by namespace and takes the $min of created_at across its entries.

Other collections at a glance

Collection	_id shape	Notes
organizations	org_id	quota.monthly_budget_usd — null = unlimited; pending_deletion_at — nullable, set by DELETE /admin/organizations/{org_id}, hard-deleted for real once past org_offboarding.grace_period_days, see Deleting an organization
users	user_id	email unique-indexed
memberships	"org:user"	role: owner/admin/member/viewer
api_keys	key_id (UUID)	key_hash unique-indexed; raw key never stored
installed_analyzers	"org:analyzer"	per-org enable/disable override
custom_rules	"org:rule_id"	one row per rule; analyzers[] — a rule can target more than one analyzer; enabled toggles inclusion without deleting; requires_review routes a match straight to needs_review regardless of LLM confidence
teams	"org:team_id"	Pure organizational grouping — no access-control effect. team_id is a slug of name, immutable after creation
team_members	"org:team_id:user_id"	Join table — a user can be on any number of teams. Deleting a team cascades: every row for that team_id is removed first (delete_team_members_for_team)
llm_credentials	"org:provider"	encrypted_key — Fernet, never returned by the API
jobs	job_id (UUID)	Platform-owned — org-scoped async work, generic over what the work is. updated_at stamped on every write (creation and every status/progress update) — what POST /admin/jobs/sweep compares against jobs.stuck_timeout_minutes. The analysis engine adds its own fields to the same document (namespace, analyzer, llm_backend/model/rule_type/retry_of backing POST /jobs/{job_id}/retry, current_resource) via JobStatus, which widens the platform’s JobRecord
usage_events	auto	TTL-indexed, drives get_monthly_llm_cost
findings	"org:hash:rule_id"	persists across re-analysis; status (incl. needs_review) never auto-resets
integrations	"org:provider"	two providers today: github (github_app — nullable, an org’s own encrypted App credentials, or null to use the platform’s config.github) and review_webhook (url, encrypted_secret — nullable, Fernet — enabled)
github_manifest_states	state_id (UUID)	Single-use CSRF token for the GitHub App manifest flow’s redirect round-trip; state_hash unique-indexed, raw token never stored, same pattern as report_links.token_hash; expires_at TTL-indexed, ~10 minutes
report_links	link_id (UUID)	token_hash unique-indexed; raw token never stored; expires_at TTL-indexed
org_invites	invite_id (UUID)	Same shape as report_links but reusable (not consumed on use) and longer-lived — invite_link_ttl_hours default 168h/7 days vs. report links’ 24h; revoked flag alongside the TTL
spans	span_id (UUID)	TTL-indexed telemetry; key_id/user_id are independent, never merged — exactly one is non-null per span
audit_events	event_id (UUID)	TTL-indexed on its own retention.audit_days — decoupled from spans’ retention.telemetry_days, see Data Retention; org_id nullable for platform-wide actions; actor_user_id/actor_key_id independent, never merged; before/after only the fields that changed, never a secret
allowed_models	"backend:model"	No org_id — platform-wide, superadmin-managed via /admin/models. input_price_per_1m_usd/output_price_per_1m_usd are manual entry (no provider exposes a pricing API); an empty collection means unrestricted (see API — Allowed models)
retention_sweep_state	fixed singleton ("retention_sweep_state")	Last retention sweep’s result — started_at/duration_ms/counts (a map keyed by sweep step, e.g. findings_purged)/error, same upsert-by-fixed-_id pattern as platform_settings/mongo_prepare_state. Backs GET /admin/retention/status
rate_limits	"policy:identity:window"	Platform-owned. Brute-force attempt counters — one document per identity per fixed window, count incremented atomically ($inc upsert, so concurrent replicas can’t lose a count and share one budget). bucket groups an identity’s windows so a successful login can clear them all; expires_at TTL-indexed. Nothing here is read after the fact — the collection is a counter, not a record. See rate_limit
job_sweep_state	fixed singleton ("job_sweep_state")	Platform-owned. Last stuck-job sweep’s result — started_at/duration_ms/jobs_marked_failed/error, same upsert-by-fixed-_id pattern as retention_sweep_state. Backs GET /admin/jobs/sweep-status

---

Indexes

Created by MongoBackend.setup(), called at application startup.

Collection	Index	Fields	Options	Covers
resources	analysis_idx	analysis asc	sparse	find_next_unanalysed
namespace_entries	org_ns_idx	org_id, namespace asc	—	find_by_namespace
namespace_entries	org_hash_idx	org_id, hash asc	—	add_to_namespace existence check
allowed_models	backend_idx	llm_backend asc	—	(advisory — lookups are by full _id, this covers a future by-backend query)
api_keys	key_hash_idx	key_hash asc	unique	auth lookup
api_keys	org_idx	org_id asc	—	list_api_keys
users	email_idx	email asc	unique	get_user_by_email
memberships	user_idx / org_idx	user_id / org_id asc	—	membership lookups
llm_credentials	org_idx	org_id asc	—	list_llm_credentials
jobs	org_idx	org_id asc	—	job lookups
jobs	status_updated_idx	status, updated_at asc	—	sweep_stuck_jobs’ cross-org scan — deliberately not scoped to a single org_id, unlike every other jobs query
usage_events	org_idx / ttl_idx	org_id / timestamp asc	TTL on timestamp	get_monthly_llm_cost, auto-expiry
findings	org_idx	org_id asc	—	list_findings
teams	org_idx	org_id asc	—	list_teams
team_members	org_team_idx / org_user_idx	(org_id, team_id) / (org_id, user_id) asc	—	list_team_members / list_all_team_memberships
integrations	org_idx / installation_idx / github_app_id_idx	org_id / installation_id / github_app.app_id asc	—	connect endpoints / webhook lookup by installation, or by App id for a custom App’s first-ever installation event
github_manifest_states	state_hash_idx / ttl_idx	state_hash asc / expires_at asc	unique / TTL on expires_at (expireAfterSeconds=0)	callback lookup; storage-cleanup backstop only — the router checks expires_at explicitly and deletes the record on first use regardless
report_links	token_hash_idx / ttl_idx	token_hash asc / expires_at asc	unique / TTL on expires_at (expireAfterSeconds=0)	token lookup; storage-cleanup backstop only — the router checks expires_at explicitly on every read, since the TTL sweep runs roughly once a minute
org_invites	token_hash_idx / org_idx / ttl_idx	token_hash asc / org_id asc / expires_at asc	unique / — / TTL on expires_at (expireAfterSeconds=0)	token lookup; org-scoped listing; same TTL-backstop reasoning as report_links
spans	ts_idx / ttl_idx / org_idx	timestamp desc / timestamp asc / org_id	TTL on timestamp (retention.telemetry_days)	span queries (newest first), auto-expiry
audit_events	org_idx / ts_idx / actor_user_idx / actor_key_idx / ttl_idx	org_id / timestamp desc / actor_user_id / actor_key_id / timestamp asc	TTL on timestamp (retention.audit_days — its own independent window, no longer shared with spans)	audit queries (newest first, org or actor filtered), auto-expiry
rate_limits	ttl_idx / bucket_idx	expires_at asc / bucket asc	TTL on expires_at (expireAfterSeconds=0)	auto-expiry of spent windows; clearing every window of one identity after a successful login. Lookups themselves are by full _id, so neither index serves a read path

Run python src/tools/init_mongo.py to create the core indexes before starting the server for the first time — setup() is idempotent and also runs the rest on every startup.

TTL indexes here are live-updatable. spans/usage_events and audit_events’s ttl_idx expireAfterSeconds isn’t fixed at index-creation time the way a TTL index normally is — PATCH /admin/settings with a changed retention_telemetry_days/retention_audit_days pushes the new value onto the already-running index via a collMod command immediately. See Data Retention.

Upgrading a database created under an older retention default

Index creation reconciles rather than fails. Mongo’s createIndex is only idempotent when the existing index matches exactly: an index of the same name with different options raises IndexOptionsConflict instead of updating it. Because startup declares every index unconditionally, a database whose ttl_idx was created under a different expireAfterSeconds would otherwise make the process crash-loop on upgrade — which is a real scenario, since audit_events gained its own retention window after previously sharing spans’.

Startup now collMods a conflicting TTL window into place and carries on, logging the change. init_mongo.py does the same when re-run against an existing database, so the tool and the app can’t drift apart.

Conflicts that aren’t a TTL window — a changed unique flag, or a changed key spec — are reported and startup stops, naming the collection, the index and the dropIndex to run. Resolving those means dropping and recreating the index, which is destructive and potentially slow on a large collection, so it’s left as an explicit operator decision rather than something the server does to your database on its own at boot.

---

Org-offboarding bulk deletes

controllers.services.org_offboarding.purge_pending_organizations cascades a plain delete_many({"org_id": org_id}) (or the equivalent single-field filter) across every org-scoped collection above — resources, namespace_entries, memberships, api_keys, installed_analyzers, custom_rules, report_templates, rule_categories, teams, team_members, llm_credentials, jobs, findings, org_invites — then the organizations document itself, last. No new indexes were needed for this: every one of those collections already carries an org_id field, and this only ever runs as a rare, admin-triggered batch operation (a scheduled org purge), not a hot request path. usage_events/audit_events are deliberately excluded from this cascade — see Platform Admin — Deleting an organization for why.

---

Query patterns

find_by_namespace(org_id, namespace)

entries = await entries_col.find({"org_id": org_id, "namespace": ns}, {"hash": 1}).to_list(None)
rks = [f"{org_id}:{e['hash']}" for e in entries]
return await resources_col.find({"_id": {"$in": rks}}, {"_id": 0}).to_list(None)

find_next_unanalysed(org_id, namespace)

entries = await entries_col.find({"org_id": org_id, "namespace": ns}, {"hash": 1}).to_list(None)
rks = [f"{org_id}:{e['hash']}" for e in entries]
return await resources_col.find_one(
    {"_id": {"$in": rks}, "analysis": None}, {"_id": 0}
)

add_to_namespace(org_id, hash, namespace) — idempotent upsert

await entries_col.update_one(
    {"_id": f"{org_id}:{namespace}:{hash}"},
    {"$setOnInsert": {"org_id": org_id, "namespace": namespace, "hash": hash}},
    upsert=True,
)

get_monthly_llm_cost(org_id) — aggregation pipeline

pipeline = [
    {"$match": {"org_id": org_id, "type": "llm_tokens", "timestamp": {"$gte": month_start}}},
    {"$group": {"_id": None, "total": {"$sum": "$cost_usd"}}},
]
result = await usage_events_col.aggregate(pipeline).to_list(length=1)

---

Configuration

storage:
  backend: "mongodb"
  mongodb:
    uri: "mongodb://localhost:27017"
    database: "mcp_analyzer"

For authenticated deployments include credentials in the URI, or inject it via the MONGODB_URI environment variable instead of the config file (recommended for production — see Configuration):

uri: "mongodb://mcp_app:changeme@mongodb:27017/mcp_analyzer?authSource=mcp_analyzer"

---

Initialisation

# First-time setup (creates collections + indexes + optional app user)
python src/tools/init_mongo.py \
  --uri mongodb://admin:secret@localhost:27017 \
  --app-user mcp_app \
  --app-password changeme

# Dry run
python src/tools/init_mongo.py --dry-run

See Tools for the full CLI reference.

---

Motor (async driver)

The backend uses Motor — the async wrapper around PyMongo.

Key difference from PyMongo:

# Motor: find() is synchronous (returns a cursor), to_list() is async
cursor = col.find({"org_id": "acme-corp"})      # sync — returns cursor immediately
docs   = await cursor.to_list(length=None)      # async — fetches documents

This is why find() calls are not awaited but to_list() calls are — the same pattern used throughout test_storage.py’s Mongo mocks.