Detection engineers are usually measured on coverage and fidelity, but they hold the key to SIEM cost too. They know which events and fields every rule depends on. That knowledge turns cost reduction from guesswork into engineering.
Why detection engineers should own cost
Most SIEMs price by ingestion volume. Every event that no detection, investigation or report uses is pure cost. Public guidance, including CISA's SIEM ingestion priorities and NIST SP 800-92 on log management, points the same way: prioritise high-value security logs and handle the rest outside the SIEM.
The core idea: value-based filtering
Sign-outs, health checks, heartbeats and duplicate events can make up a large share of SIEM volume while adding almost nothing to detection. Value-based filtering keeps high-fidelity security events in the SIEM and sends low-risk data to cheaper storage, without deleting anything you might need later.
Six techniques that reduce ingestion safely
- Filter low-value events. Drop routine sign-outs, heartbeats and irrelevant metadata before ingestion, and archive the full events in cold storage for compliance.
- Summarise high-frequency activity. Roll up successful authentications, DNS and proxy flows into counts with first and last timestamps, and keep exceptions and outliers at full detail.
- Prune unused fields. Map each detection to the fields it reads and drop the rest. A common schema such as OCSF or ECS makes this repeatable.
- Tier retention. Hot in the SIEM for 0–30 days, warm in a security data lake for 30–90 days, cold in object storage for up to seven years.
- Route by purpose. Identity, endpoint, network security and cloud control-plane events go to the SIEM; verbose operational logs go to log management.
- Place data deliberately. Keep authentication failures, MFA challenges, EDR and IDS/IPS alerts and cloud IAM changes hot. Move summarised successes, debug logs, health checks and compliance copies out.
A 30-day implementation plan
- Week 1: baseline GB/day and cost by source, and identify the top cost drivers.
- Week 2: map every detection rule to the sources and fields it uses, and flag data no rule references.
- Week 3: deploy filtering, deduplication and enrichment in your security data pipeline, under version control.
- Week 4: move non-critical and compliance data to warm or cold storage, and publish recall runbooks.
- Ongoing: track cost per day, alert fidelity, false-positive rate and MTTD before and after.
Frequently asked questions
Does filtering reduce the quality of incident investigations?
No. Full-fidelity logs are archived on low-cost storage and can be searched or recalled into the SIEM when an investigation needs them. Only the copy sent to the SIEM is reduced.
How do we make sure detection coverage stays intact?
Maintain a MITRE ATT&CK-mapped detection catalogue, keep every field it references unfiltered and compare coverage and alert quality before and after each change.