SIEM pricing grows with data volume, and log volume grows every year. Sending everything to the SIEM and keeping it there for years is the fastest way to turn a security budget into a storage bill. A tiered storage architecture breaks that link: each log lands in the tier that matches how it's actually used.
The problem with "everything in the SIEM"
Most SIEM deployments start by ingesting every source at full fidelity and keeping it hot for the full retention period. Over time three things happen:
- Cost scales with volume, not value. Verbose sources such as DNS, proxy, flow and successful authentications dominate GB/day while contributing little to detections.
- Retention is paid at hot-storage prices. Compliance logs that are queried a few times a year sit on the most expensive storage you own.
- Analysts wade through noise. Searches and correlation slow down as indexes grow.
A three-tier model: hot, warm and cold
Tier by use, not by vendor. The question for every source is: what do we do with this data, and how fast do we need it?
| Tier | Typical window | Purpose | Typical platform |
|---|---|---|---|
| Hot | 0–30 days | Real-time detection, alerting, active investigations | Your SIEM (Splunk, Sentinel, Elastic, Google SecOps, QRadar) |
| Warm | 30–90 days | Threat hunting, forensic search, investigation look-back | Security data lake or search tier: ClickHouse, Elasticsearch, OpenSearch, Azure Data Explorer |
| Cold | 90 days to 7+ years | Compliance archive, legal hold, rare forensic recall | Object storage: Amazon S3, Azure Data Lake Storage, Google Cloud Storage, MinIO |
Design principles that protect detection
- Detection-critical data stays hot. Any field or event referenced by a detection rule or an ATT&CK-mapped use case is routed to the SIEM, full stop.
- Cold means full fidelity. The archive keeps the raw, unfiltered event so you can reconstruct anything later. Filtering decides what reaches the SIEM, not what you keep.
- Recall is designed and tested before migration. Document how warm and cold data is searched or rehydrated into the SIEM, and test it with the SOC before you move a single source.
- The pipeline does the work. Filtering, deduplication, enrichment and routing happen in a security data pipeline before data reaches any tier, so storage tiers stay simple.
What changes when you tier
- 30–70% lower SIEM ingestion on typical source mixes, depending on how much verbose data you send today.
- No loss of detection coverage, because detection-critical data never leaves the hot tier.
- Multi-year retention at object-storage cost, in open formats such as Parquet and Apache Iceberg.
- Around 90 days of searchable forensic depth in the warm tier for hunts and investigations.
Implementation in four phases
- Design: inventory sources, measure GB/day and cost per source, and map each source and field to hot, warm or cold based on detection and investigation value.
- Build: configure pipeline routes, storage tiers, lifecycle policies and immutability for the archive.
- Deploy: migrate progressively, starting with high-volume, low-detection-value sources, and compare alert volume and coverage before and after each step.
- Run: publish runbooks for recall, review routing quarterly and update it whenever detections or regulations change.
Frequently asked questions
How long does it take to recall data from warm or cold storage?
Warm-tier searches typically return in seconds to minutes. Cold-tier recall into the SIEM usually takes minutes to hours, depending on volume and storage class. That's why recall procedures are documented and tested up front.
Does tiered storage depend on a specific SIEM?
No. Routing happens in the pipeline layer, so the same model works with Splunk, Microsoft Sentinel, Elastic, Google SecOps, QRadar or OpenSearch, and survives a future SIEM migration.
How do we keep the archive tamper-proof for compliance?
Use immutable storage policies such as object lock, restrict and log access to the archive, and record hashes at write time so integrity can be verified during audits.