Building SENTINAL: How I Built an AI-Powered WAF with XGBoost, Gemini, and 5 Microservices in 36 Hours
A deep technical breakdown of SENTINAL — an AI-powered Web Application Firewall built at HackByte 4.0. Learn how XGBoost detects attacks in real time, how a Gemini-powered policy agent makes autonomous security decisions, and why "human-in-the-loop" enforcement is the right architecture for irreversible actions.
What you'll learn
- 01Pattern matching + XGBoost hybrid detection is more robust than either approach alone — patterns become feature signals, not gatekeepers
- 02PolicyGuard-v1's core rule: auto-execute reversible actions, always queue irreversible ones — human intent required for permanent bans and shutdowns
- 03Fire-and-forget async calls to Nexus kept Gateway latency near-zero — secondary AI reasoning never blocks the primary request path
- 0430-second in-memory blocklist cache eliminated MongoDB load on the hottest operation in the entire system
- 05Socket.IO push events replaced 5-second polling — real-time security data requires real push, not fake-real polling
- 06Streaming Gemini responses via SSE transforms user perception from "waiting" to "watching the AI think"
- 07The sentinal-middleware npm package proves hackathon code can graduate into reusable open-source libraries
- 08Human-in-the-loop enforcement is not a UX feature — it is an architectural guarantee that irreversible AI decisions require human consent
Traditional firewalls block what they've seen before. SENTINAL blocks what it understands. Here's exactly how we built it — every architectural decision, every failure, every lesson.
#Why Another Firewall?
Most developers think of a Web Application Firewall (WAF) as something managed — Cloudflare, AWS WAF, ModSecurity. You flip it on, set some rules, and forget it. But when you look under the hood, these tools are fundamentally pattern matchers. They maintain giant lists of known bad strings — ' OR 1=1--, <script>, ../../../etc/passwd — and block requests that match.
This approach has two deep problems:
- Evasion is trivial.
%27%20OR%201%3D1--,<ScRiPt>,.././.././etc/passwd— URL encoding, case mutation, Unicode tricks. A pattern matcher that doesn't understand the meaning of a request is always one obfuscation layer away from being bypassed. - Zero forensic value. When your WAF blocks a request, you get a log line. You don't get: Why was this dangerous? What was the attacker trying to achieve? Is this the third attempt from this IP in 10 minutes? Does this correlate with suspicious activity from two other IPs?
The idea behind SENTINAL was: what if the firewall could understand what it's seeing, reason about the right response, and explain its decisions in plain English?
#The Architecture: 5 Services, One Purpose
Before writing a single line of code at HackByte 4.0, we spent 90 minutes on a whiteboard. The result was a microservice architecture with a clear single-responsibility principle per service:
1INCOMING TRAFFIC
2 │
3 ▼
4Gateway (Node.js :3000) ← the only public-facing service
5 │
6 ┌──┼──────────────────┐
7 ▼ ▼ ▼
8Detection MongoDB Nexus Agent
9Engine Atlas (Python :8004)
10(Python (7 collections)
11 :8002)
12 │
13 ▼
14React Dashboard (:5173) ← Socket.IO real-time feed
15Each service is in the language best suited for it:
- Node.js for the gateway — fast I/O, Socket.IO ecosystem, Express middleware model
- Python for ML and AI — scikit-learn, XGBoost, FastAPI, Google Gemini SDK all first-class
- React for the dashboard — component model, real-time state management
This separation meant we could fail and restart individual services without taking down the whole system.
#//Service 1: The Gateway — More Than a Reverse Proxy
The Gateway is the only service that touches incoming traffic. Every HTTP request to a protected application first passes through it. Here's the exact middleware chain:
1// server.js — middleware order matters
2app.use(cors());
3app.use(express.json());
4app.use(blocklistMiddleware); // ← IP check FIRST
5app.use(requestLogger); // ← then log
6// routes registered after#//The Blocklist Middleware
The most important middleware runs first — before any route handler, before any logging:
1// blocklistMiddleware.js
2let cache = [];
3let cacheTime = 0;
4const CACHE_TTL = 30_000; // 30 seconds
5
6async function blocklistMiddleware(req, res, next) {
7 const now = Date.now();
8 if (now - cacheTime > CACHE_TTL) {
9 cache = await BlockedIP.find({ expiresAt: { $gt: new Date() } });
10 cacheTime = now;
11 }
12 const blocked = cache.find(b => b.ip === req.ip);
13 if (blocked) return res.status(403).json({ error: 'Blocked', reason: blocked.reason });
14 next();
15}The 30-second in-memory cache is a critical design decision. Without it, every single HTTP request would hit MongoDB — at any meaningful traffic volume, that's a database bottleneck on your most latency-sensitive operation. The trade-off: a newly blocked IP can still make requests for up to 30 seconds after being added to the blocklist. For a WAF, that's acceptable — this isn't a real-time nanosecond system.
#//The Detection Call
After logging the request, the Gateway calls the Detection Engine:
1// attackService.js
2const response = await axios.post(`${DETECTION_URL}/detect`, {
3 method: req.method,
4 url: req.url,
5 body: req.body,
6 headers: req.headers,
7 ip: req.ip
8});
9
10const { detected, attackType, severity, confidence } = response.data;
11
12if (detected) {
13 await reportAttack({ attackType, severity, confidence, ip, url });
14}Notice the Nexus call is fire-and-forget — we await saving the attack, then call Nexus without awaiting it. This keeps Gateway response latency near-zero regardless of how long Nexus takes to reason.
#//Service 2: The Detection Engine — XGBoost Meets FastAPI
This is where the ML lives. The Detection Engine is a Python FastAPI app that accepts a raw HTTP request object and returns a classification.
#//Why XGBoost Over a Neural Network?
This question comes up every time someone sees "ML" and "security" in the same sentence. Here's the honest comparison:
For a real-time WAF that needs to classify every HTTP request in milliseconds on a CPU-only server, XGBoost is the right choice. The slight accuracy disadvantage on highly obfuscated attacks is handled by the Gemini layer.
#//The Detection Pipeline
1# app/detector.py
2def detect(request_data: RequestSchema) -> DetectionResult:
3 # Step 1: Feature extraction
4 features = extract_features(request_data)
5 # → URL length, special char density, keyword presence,
6 # body size, header anomalies, method/path combination
7
8 # Step 2: Pattern pre-filter (fast, zero-cost)
9 pattern_match = run_patterns(request_data)
10 if pattern_match.matched:
11 # Don't skip ML — use pattern as feature signal
12 features['pattern_flag'] = 1
13 features['pattern_type'] = pattern_match.attack_type
14
15 # Step 3: XGBoost inference
16 proba = model.predict_proba([features])[0]
17 confidence = float(max(proba))
18 attack_class = CLASSES[np.argmax(proba)]
19
20 return DetectionResult(
21 detected=confidence > THRESHOLD,
22 attackType=attack_class,
23 severity=map_severity(confidence),
24 confidence=confidence
25 )The pattern matching doesn't replace ML — it's a feature signal fed into the model. A URL containing UNION SELECT is more likely to be an SQLi attempt, and that information should be available to the classifier.
#//The 11 Attack Classes
1sqli → SQL Injection
2xss → Cross-Site Scripting
3traversal → Path/Directory Traversal
4cmd_injection → OS Command Injection
5brute_force → Brute Force / Credential Stuffing
6ssrf → Server-Side Request Forgery
7lfi_rfi → Local/Remote File Inclusion
8xxe → XML External Entity
9webshell → Webshell Upload Attempt
10hpp → HTTP Parameter Pollution
11unknown → Anomalous but unclassified#Geo-IP + AbuseIPDB Enrichment
A late addition to the Detection Engine was threat intelligence enrichment:
1# Runs async alongside detection, doesn't block the response
2async def enrich_ip(ip: str) -> dict:
3 geo = await httpx.get(f"http://ip-api.com/json/{ip}")
4 abuse = await httpx.get(
5 "https://api.abuseipdb.com/api/v2/check",
6 headers={"Key": ABUSEIPDB_API_KEY},
7 params={"ipAddress": ip, "maxAgeInDays": 90}
8 )
9 return {
10 "country": geo.json().get("country"),
11 "abuseScore": abuse.json()["data"]["abuseConfidenceScore"]
12 }An IP with an AbuseIPDB confidence score > 80 gets its threat severity bumped up — even if the ML model returns medium confidence.
#//Service 3: The Nexus Agent — AI That Knows What It Can't Do
Nexus is the most architecturally interesting service. It's a Python FastAPI agent that receives a confirmed attack and decides what to do about it. The key insight is the PolicyGuard-v1 rule system — a tiering of actions by reversibility:
1# nexus/policy.py
2POLICY_RULES = {
3 "RULE_001": {
4 "action": "shutdown_endpoint",
5 "disposition": "BLOCK", # Always queue — never auto-execute
6 "reason": "Irreversible — requires human authorization"
7 },
8 "RULE_002": {
9 "action": "permanent_ban_ip",
10 "disposition": "BLOCK", # Always queue
11 "reason": "Permanent action — requires human authorization"
12 },
13 "RULE_003": {
14 "action": "rate_limit_ip",
15 "disposition": "AUTO", # Execute immediately
16 "reason": "Reversible — TTL will expire"
17 },
18 "RULE_004": {
19 "action": "send_alert",
20 "disposition": "AUTO"
21 },
22 "RULE_005": {
23 "action": "log_attack",
24 "disposition": "AUTO"
25 }
26}The core principle: if an action can be undone, automate it. If it can't, require a human.
A rate limit expires. A permanent IP ban doesn't. Shutting down an endpoint takes the application offline. These decisions need human intent — not AI confidence.
#//Gemini as a Reasoning Layer
Nexus uses Gemini 1.5 Pro not for detection, but for reasoning about the decision
1async def get_gemini_reasoning(attack_data: dict) -> str:
2 prompt = f"""
3 Analyze this security event and recommend a response action.
4
5 Attack: {attack_data['attackType']}
6 Severity: {attack_data['severity']}
7 Confidence: {attack_data['confidence']}
8 IP: {attack_data['ip']}
9 URL targeted: {attack_data['url']}
10
11 Choose ONE action: rate_limit_ip | permanent_ban_ip | flag_for_review
12 Explain your reasoning in 2 sentences.
13 """
14 response = await gemini_model.generate_content_async(prompt)
15 return response.textThis reasoning is stored with the queued action — so when a human sees a permanent_ban_ip proposal in the action queue, they also see why Nexus recommended it.
#//Service 4: The PCAP Processor — Offline Forensics
The PCAP Processor handles a different use case: you have a .pcap network capture file (from Wireshark, tcpdump) and want to know if there were any attacks in it.
1# pcap_processor/app/analyzer.py
2from scapy.all import rdpcap, TCP, IP
3
4def analyze_pcap(filepath: str) -> list[AttackCandidate]:
5 packets = rdpcap(filepath)
6 flows = build_flows(packets) # Group by src_ip:dst_ip:dst_port
7
8 candidates = []
9 for flow in flows:
10 # Check SYN flood: many SYN, few SYN-ACK
11 if is_syn_flood(flow, threshold=SYN_FLOOD_PPS_THRESHOLD):
12 candidates.append(AttackCandidate(type="syn_flood", ...))
13
14 # Check port scan: one IP → many ports
15 if is_port_scan(flow, threshold=PORT_SCAN_THRESHOLD):
16 candidates.append(AttackCandidate(type="port_scan", ...))
17
18 # Check HTTP payload for WAF attack patterns
19 http_attacks = inspect_http_payloads(flow)
20 candidates.extend(http_attacks)
21
22 return candidatesAll thresholds are tunable via environment variables — PORT_SCAN_THRESHOLD=15, SYN_FLOOD_PPS_THRESHOLD=100, DDOS_PPS_THRESHOLD=500 — because what looks like a DDoS on a personal blog is normal traffic on an enterprise API.
#The MongoDB Schema Design
Seven collections, each with a clear purpose:
1// BlockedIP.js — the most performance-critical schema
2const BlockedIPSchema = new Schema({
3 ip: { type: String, required: true, index: true },
4 reason: { type: String, required: true },
5 attackType: { type: String },
6 blockedAt: { type: Date, default: Date.now },
7 expiresAt: { type: Date }, // ← TTL index on this field
8 blockedBy: { type: String } // 'nexus-auto' | 'human-dashboard'
9});
10BlockedIPSchema.index({ expiresAt: 1 }, { expireAfterSeconds: 0 });
11// MongoDB's TTL index automatically deletes documents when expiresAt passes
12
13// AttackEvent.js — the richest schema
14const AttackEventSchema = new Schema({
15 ip: { type: String, required: true },
16 attackType: { type: String, required: true },
17 severity: { type: String, enum: ['low','medium','high','critical'] },
18 confidence: { type: Number, min: 0, max: 1 },
19 method: String,
20 url: String,
21 payload: String, // sanitized snapshot of the malicious payload
22 geo: { country: String, city: String, abuseScore: Number },
23 timestamp: { type: Date, default: Date.now }
24});The AuditLog collection has no update or delete route exposed in the API — by design. Every approve/reject action appends a new document. This is your immutable record of every security decision ever made.
#//Real-Time with Socket.IO: Push Over Poll
The first version of the dashboard used setInterval polling every 5 seconds. The problem: 5-second-old attack data in a live security dashboard is useless, and polling creates a constant DB load even when nothing is happening.
The fix was Socket.IO with 4 named events:
1// backend/src/sockets/broadcast.js
2const EVENTS = {
3 ATTACK_NEW: 'attack:new',
4 ALERT_NEW: 'alert:new',
5 ACTION_PENDING: 'action:pending',
6 BLOCKLIST_UPDATED: 'blocklist:updated'
7};
8
9// Called inside attackService.js after saving to MongoDB
10function broadcastAttack(io, attackEvent) {
11 io.emit(EVENTS.ATTACK_NEW, attackEvent);
12
13 if (['high','critical'].includes(attackEvent.severity)) {
14 io.emit(EVENTS.ALERT_NEW, { ...attackEvent, alertType: 'severity' });
15 }
16}On the frontend, a single custom hook handles all real-time subscriptions:
1// dashboard/src/hooks/useSocket.js
2export function useSocket() {
3 const [socket] = useState(() => io(GATEWAY_URL));
4
5 useEffect(() => {
6 socket.on('attack:new', (data) => {
7 queryClient.setQueryData(['attacks'], prev => [data, ...prev]);
8 });
9 socket.on('action:pending', (data) => {
10 queryClient.setQueryData(['actions'], prev => [data, ...prev]);
11 });
12 return () => socket.disconnect();
13 }, []);
14
15 return socket;
16}Every page that needs real-time data calls useSocket() — one connection, one hook, shared across the app.
#//The AI Copilot — Streaming Gemini Responses
The /copilot page (21KB, the second largest page) is a security-aware AI assistant. What makes it interesting is the streaming implementation:
1// backend/src/routes/gemini.js
2router.get('/chat/stream', async (req, res) => {
3 const { message, history } = req.query;
4
5 res.setHeader('Content-Type', 'text/event-stream');
6 res.setHeader('Cache-Control', 'no-cache');
7
8 // Inject live attack context into the system prompt
9 const recentAttacks = await AttackEvent.find().sort('-timestamp').limit(10);
10 const context = `Recent attacks: ${JSON.stringify(recentAttacks)}`;
11
12 const stream = await geminiModel.generateContentStream({
13 contents: [
14 { role: 'user', parts: [{ text: context }] },
15 ...parseHistory(history),
16 { role: 'user', parts: [{ text: message }] }
17 ]
18 });
19
20 for await (const chunk of stream) {
21 res.write(`data: ${JSON.stringify({ type: 'chunk', text: chunk.text() })}\n\n`);
22 }
23
24 res.write(`data: ${JSON.stringify({ type: 'done' })}\n\n`);
25 res.end();
26});The key insight: Gemini knows about your actual attack data because we inject recent attack events into the system prompt. When you ask "Which IP is most aggressive today?" — Gemini has the actual data to answer it.
#//The sentinal-middleware npm Package
One of the most practical outcomes of this project was extracting the rate limiting logic into a standalone package:
1// sentinal-middleware/src/rateLimiter.js
2const rateLimit = require('express-rate-limit');
3
4const ingestLimiter = rateLimit({
5 windowMs: 60 * 1000, // 1 minute
6 max: 100, // 100 requests per IP
7 standardHeaders: true,
8 legacyHeaders: false,
9 message: { error: 'Too many requests', retryAfter: 60 }
10});
11
12const globalLimiter = rateLimit({
13 windowMs: 60 * 1000,
14 max: 300
15});
16
17module.exports = { ingestLimiter, globalLimiter };Any Express app can now use SENTINAL's battle-tested rate limiting in two lines:
1const { globalLimiter, validate } = require('sentinal-middleware');
2app.use(globalLimiter);The package is MIT licensed, has express >=4.0.0 as a peer dependency, and is fully structured for npm publish. The lesson here: hackathon code doesn't have to die after the hackathon. Identify the pieces that are genuinely reusable and extract them.
#The Human-in-the-Loop Design: Why It Won the Judges Over
The feature that generated the most discussion during judging wasn't the ML model or the Gemini integration. It was the /action-queue page — where a human approves or rejects AI-proposed actions.
Here's the philosophical argument for it:
A permanent_ban_ip action is irreversible. If the IP belongs to a legitimate user behind a NAT router, a shared office Wi-Fi, or a corporate proxy — banning it locks out potentially thousands of innocent users. An AI model with 97% confidence is still wrong 3% of the time. At scale, that 3% matters enormously.
The architecture enforces this at the policy level — not as a UI toggle that can be disabled:
1# nexus/policy.py — this check cannot be bypassed
2if action in ['permanent_ban_ip', 'shutdown_endpoint']:
3 disposition = 'BLOCK' # Always queue, always
4 post_to_action_queue(action, reasoning)
5 return # Never fall through to auto-executeThe action queue shows: what action is proposed, what IP it targets, what attack triggered it, and Gemini's written reasoning. A human reads it in 5 seconds and clicks Approve or Reject.
This pattern — AI proposes, human disposes — is the right architecture for any system where AI decisions have real-world consequences that can't be undone.
#//What I'd Do Differently
- 1Establish service contracts before writing code. We lost ~3 hours to incompatible request/response shapes between the Gateway and Detection Engine. A 10-minute OpenAPI spec per service would have prevented this entirely.
2. Use Redis for rate limiter state.
The current sentinal-middleware implementation stores rate limit counters in process memory. If the Gateway restarts, all counters reset. A Redis-backed store (redis option in express-rate-limit) would persist state across restarts and support horizontal scaling.
3. Model versioning from day one.
The XGBoost model is a static .joblib file. There's no versioning, no rollback, no A/B testing between model versions. A proper ML serving setup would have model version metadata stored alongside each attack prediction.
4. Push vs. pull decision on day one. We started with polling and switched to Socket.IO mid-hackathon. This forced a significant frontend refactor. Decide your real-time strategy before writing frontend pages.
5. The demo-target is a liability. An intentionally vulnerable Express app in the same repository as your WAF is a documentation/communication risk. Keep it clearly isolated, clearly documented, and never deployed on the same server as anything real.
Key Takeaways
- ✓Pattern matching + XGBoost hybrid detection is more robust than either approach alone — patterns become feature signals, not gatekeepers
- ✓PolicyGuard-v1's core rule: auto-execute reversible actions, always queue irreversible ones — human intent required for permanent bans and shutdowns
- ✓Fire-and-forget async calls to Nexus kept Gateway latency near-zero — secondary AI reasoning never blocks the primary request path
- ✓30-second in-memory blocklist cache eliminated MongoDB load on the hottest operation in the entire system
- ✓Socket.IO push events replaced 5-second polling — real-time security data requires real push, not fake-real polling
- ✓Streaming Gemini responses via SSE transforms user perception from "waiting" to "watching the AI think"
- ✓The sentinal-middleware npm package proves hackathon code can graduate into reusable open-source libraries
- ✓Human-in-the-loop enforcement is not a UX feature — it is an architectural guarantee that irreversible AI decisions require human consent