Cloud-Native Intrusion Detection Pipeline
University of London · MSc Thesis
Challenge: Cloud-native networks produce high-volume telemetry that traditional IDS cannot process in real time without expensive SIEM licensing. SMEs need a low-cost, open-source alternative.
What I built: A five-stage containerised pipeline: Zeek parses live traffic into structured JSON, Redis Streams buffers telemetry with automatic recovery, and a calibrated XGBoost model scores each flow for anomalies before a policy engine decides the response.
Outcome: 84.2% ROC-AUC on the CICIDS2017 benchmark. After calibrating for a real-world false-alarm budget, the pipeline catches 40% of attacks while keeping false positives below 9%. Every step is fully documented and reproducible on GitHub.