Problem
Log pipeline economics model that routes each telemetry source to index, field-filter, archive, or drop, then reports monthly ingest cost against value-weighted detection coverage and names exactly which detections each saving gives up.
SIEMLog EngineeringCost OptimisationDetection CoverageObservabilitySecurity OperationsCase study / interactive demo
Case study
From problem to working system
My role
Security engineer and full-stack developer
Solution
Log pipeline economics model that routes each telemetry source to index, field-filter, archive, or drop, then reports monthly ingest cost against value-weighted detection coverage and names exactly which detections each saving gives up.
Architecture
The implementation combines the following technologies and system concerns.
SIEMLog EngineeringCost OptimisationDetection CoverageObservabilitySecurity Operations
How it was built
- Modelled ingest cost per source at realistic per-gigabyte pricing across four routing tiers, since indexing everything is a decision most teams make once and never revisit.
- Modelled field-level filtering at the collector as a distinct tier, because keeping the queried fields retains most detection value at a fraction of the cost.
Security decisions
No project-specific security control is documented in the current project record. The case study avoids claiming controls that were not verified.
Major challenges
- Weighted each source by the detections that depend on it, so coverage loss is expressed in detections rather than in gigabytes.
- Named the specific detections given up by archiving or dropping a source, which is the disclosure that makes a cost decision reviewable.
- Highlighted sources being fully indexed that no detection depends on, which is where the first savings always are.
Verified evidence
Results and measurable impact
- Named the specific detections given up by archiving or dropping a source, which is the disclosure that makes a cost decision reviewable.
- Modelled field-level filtering at the collector as a distinct tier, because keeping the queried fields retains most detection value at a fraction of the cost.
- Highlighted sources being fully indexed that no detection depends on, which is where the first savings always are.
- Reported cost and coverage side by side so the tradeoff is explicit rather than discovered after the detections stop firing.
No separate numeric outcome is documented, so this section shows shipped technical evidence without inventing metrics.
Screenshots and access
Product view
Interactive Demo
A scoped, fully functional recreation of this project's core feature runs below, live in your browser. Reset it, resize it, or expand it to full screen.
siem-cost.zainkhalilkhan.com
SIEM Cost
Security platform
Sandbox telemetry
SIEM CostWorkspace3 updates
SIEM Cost · Ingest Routingdetection coverage retained at the current routing
$111k/mo · 100%
Every SIEM bill is a routing decision that someone made once and never revisited. Each source can be indexed, filtered to the fields detections actually use, archived cheaply, or dropped. The point is not to minimise cost, it is to see exactly which detections you are giving up for each dollar saved.
Monthly ingest cost
$111,240
baseline $111,240
Saved
$0
0% off baseline
Daily volume
1545 GB
across 7 sources
Detections lost
0
none
Coverage0 detections lost
100%retained
Monthly spend per source$111,240 of $111,240 baseline
Cost against fully-indexed baselinesaved $0
Every detection still has its source. This routing saves $0 a month without giving anything up.
Source routing
Zain Khalil Khan
Client-side sandbox. State is in memory and nothing is sent to a server.