Skip to content

v12 - feat(plugins/rule-flood-guard): auto-disable flooding correlation rules and notify via bell#2354

Merged
yllada merged 4 commits into
release/v12.0.0from
backlog/rule-flood-guard
Jul 19, 2026
Merged

v12 - feat(plugins/rule-flood-guard): auto-disable flooding correlation rules and notify via bell#2354
yllada merged 4 commits into
release/v12.0.0from
backlog/rule-flood-guard

Conversation

@yllada

@yllada yllada commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

What

Adds a new standalone plugin (plugins/rule-flood-guard) that automatically disables correlation rules generating alert fatigue, and notifies the user through the notification bell.

When a rule produces more than 50 open, non-deduplicated, top-level alerts within a rolling 24-hour window, the guard:

  1. Disables the rule via the existing enable/disable mechanism (no new UI, no new user-facing toggle).
  2. Sends exactly one notification instructing the user to either mark the alerts as false positives (existing Alert Tag Rules feature) or fine-tune the rule (deduplicateBy/groupBy).

Why

Some correlation rules, when misconfigured or missing deduplicateBy/groupBy, flood the alerts list with individual, non-deduplicated alerts instead of grouping them. This causes alert fatigue: analysts stop paying attention to the alerts list and miss real signal. This is a safety net, not a fix for the root dedup-configuration problem (out of scope — see below).

Changes

Area Change
plugins/rule-flood-guard/ (new) Standalone Go plugin: ticker-based background job, OpenSearch aggregation (count per rule, 24h window), calls backend to disable + notify. Config is file-based and hot-reloadable (system_plugins_rule-flood-guard.yaml), no restart needed to tune threshold/window/interval.
backend/modules/eventprocessing/ New internal-key-gated endpoint (PUT /internal/correlation-rule/deactivate) so the plugin can disable a rule by name.
backend/modules/notifications/ Added SYSTEM as a valid notification source (the enum value already existed but wasn't wired into request validation).
.github/workflows/v12-deployment-pipeline.yml Added the plugin to the CI build matrix.

Testing

Manually verified end-to-end against a local OpenSearch + Postgres + backend: created a real correlation rule, seeded 51 alerts, confirmed the guard disabled the rule (changed:true), confirmed a retry was a no-op (changed:false, no duplicate notification), confirmed the notification appeared with the correct message. No automated tests added, matching this project's current test coverage convention.

@github-actions

Copy link
Copy Markdown

🛑 AI review — Sensitive area, extra care recommended

This PR touches critical paths or introduces changes the model cannot judge with sufficient confidence. Review carefully before merging.

⚠️ architecture (gemini-3-flash-lite) — minor findings

Summary: Introduces a new internal API endpoint for rule deactivation and a new plugin; logic is sound but requires minor cleanup.

  • medium backend/modules/eventprocessing/handler/correlation_rule_internal.go:45 — Manual filtering of rules in the handler layer. This logic should be encapsulated within the usecase layer to maintain clean separation of concerns.
  • low backend/modules/notifications/domain/enums.go:21 — Adding 'SourceSystem' to enums is correct, but ensure that downstream consumers (e.g., UI or reporting modules) are updated to handle this new source type.
  • low plugins/rule-flood-guard/go.mod:3 — Go version 1.25.5 is specified, which is significantly ahead of standard stable releases. Ensure this matches the project's global Go toolchain version.

🛑 bugs (gemini-3-flash-lite) — high/critical — please review

Summary: Missing resource cleanup in backend client and potential race condition in config holder.

  • high plugins/rule-flood-guard/backend.go:50 — Missing response body closure on error path. If resp is not nil but status code is >= 400, the body is read but never closed, leading to connection leaks.
  • medium plugins/rule-flood-guard/config.go:104 — The setKnobs method updates individual fields of the struct while holding a lock, but the Config struct is passed by value in Get(). While not a data race, it is inconsistent; consider returning a copy or using an atomic value for the entire config object.
  • low plugins/rule-flood-guard/README.md:1 — Typo: 'UTMStack' is likely intended to be 'UTMStack' (consistent with project name), but ensure consistency across documentation. Also, 'non-deduplicated' is grammatically awkward; consider 'not deduplicated'.

🛑 security (gemini-3-flash-lite) — high/critical — please review

Summary: Introduction of internal API endpoint for rule deactivation and associated plugin logic.

  • high backend/modules/eventprocessing/handler/correlation_rule_internal.go:20 — New internal API endpoint /eventprocessing/internal/correlation-rule/deactivate allows arbitrary rule deactivation. Ensure the internal authentication middleware (RequireInternal) is robust and cannot be bypassed via header spoofing or misconfiguration.
  • medium plugins/rule-flood-guard/backend.go:45 — The plugin uses a custom header 'X-Internal-Key' for authentication. Ensure this key is rotated and handled securely, as it grants administrative control over correlation rules.

🟢 go-deps — up to date

No pending Go dependency updates.

@yllada yllada linked an issue Jul 19, 2026 that may be closed by this pull request
@yllada
yllada merged commit 94ba22c into release/v12.0.0 Jul 19, 2026
1 check passed
@yllada
yllada deleted the backlog/rule-flood-guard branch July 19, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v12 - Auto-disable flooding correlation rules and notify via bell

1 participant