Nuclei Scanner High Availability HA Setup

Nuclei Scanner HA

Nuclei Scanner High Availability Distributed Scanning Queue Redis RabbitMQ Docker Kubernetes Workers Template CVE Vulnerability CI/CD DevSecOps

Nuclei Distributed Architecture

# === Nuclei HA Architecture ===

# Architecture:
# ┌──────────┐     ┌─────────┐     ┌──────────┐
# │Dispatcher├────►│  Redis   ├────►│ Worker 1 │
# │(Targets) │     │  Queue   │     │ (Nuclei) │
# └──────────┘     │          │     └────┬─────┘
#                  │          │          │
#                  │          ├────►┌────┴─────┐
#                  │          │     │ Worker 2 │
#                  │          │     │ (Nuclei) │
#                  │          │     └────┬─────┘
#                  │          │          │
#                  │          ├────►┌────┴─────┐
#                  └──────────┘     │ Worker N │
#                                  └────┬─────┘
#                                       │
#                              ┌────────┴────────┐
#                              │ Results Collector│
#                              │ (Elasticsearch)  │
#                              └─────────────────┘

# Python — Dispatcher (Redis Queue)
# import redis
# import json
#
# r = redis.Redis(host='redis', port=6379)
#
# def dispatch_targets(targets_file, templates):
#     with open(targets_file) as f:
#         targets = [line.strip() for line in f if line.strip()]
#
#     for target in targets:
#         job = {
#             "target": target,
#             "templates": templates,
#             "severity": "critical, high, medium",
#             "retries": 0,
#         }
#         r.lpush("nuclei:jobs", json.dumps(job))
#     print(f"Dispatched {len(targets)} targets")
#
# dispatch_targets("targets.txt", ["cves/", "exposures/", "misconfigurations/"])

# Python — Worker
# import subprocess
# import redis
# import json
#
# r = redis.Redis(host='redis', port=6379)
#
# while True:
#     _, job_data = r.brpop("nuclei:jobs")
#     job = json.loads(job_data)
#
#     cmd = [
#         "nuclei",
#         "-u", job["target"],
#         "-t", job["templates"],
#         "-severity", job["severity"],
#         "-json", "-o", "/tmp/result.json",
#     ]
#
#     result = subprocess.run(cmd, capture_output=True, timeout=300)
#
#     if result.returncode == 0:
#         with open("/tmp/result.json") as f:
#             findings = f.read()
#         r.lpush("nuclei:results", findings)
#     else:
#         if job["retries"] < 3:
#             job["retries"] += 1
#             r.lpush("nuclei:jobs", json.dumps(job))

from dataclasses import dataclass

@dataclass
class ScanWorker:
    worker_id: str
    status: str
    current_target: str
    scans_completed: int
    findings: int
    uptime_hrs: float

workers = [
    ScanWorker("worker-01", "Scanning", "api.example.com", 145, 23, 48.5),
    ScanWorker("worker-02", "Scanning", "web.example.com", 132, 18, 48.5),
    ScanWorker("worker-03", "Idle", "—", 140, 21, 48.5),
    ScanWorker("worker-04", "Scanning", "mail.example.com", 128, 15, 24.2),
    ScanWorker("worker-05", "Error", "cdn.example.com", 95, 12, 12.1),
]

print("=== Scan Workers ===")
for w in workers:
    print(f"  [{w.status}] {w.worker_id}: {w.current_target}")
    print(f"    Completed: {w.scans_completed} | Findings: {w.findings} | Uptime: {w.uptime_hrs}h")

Docker และ Kubernetes

# === Docker Swarm / Kubernetes ===

# Docker Compose — Nuclei HA
# services:
#   redis:
#     image: redis:7-alpine
#     ports: ["6379:6379"]
#
#   dispatcher:
#     build: ./dispatcher
#     depends_on: [redis]
#     volumes: [./targets:/targets]
#     environment:
#       REDIS_HOST: redis
#
#   worker:
#     build: ./worker
#     depends_on: [redis]
#     deploy:
#       replicas: 5
#       resources:
#         limits: { cpus: "1", memory: "512M" }
#     environment:
#       REDIS_HOST: redis
#
#   collector:
#     build: ./collector
#     depends_on: [redis, elasticsearch]
#     environment:
#       REDIS_HOST: redis
#       ES_HOST: elasticsearch
#
#   elasticsearch:
#     image: elasticsearch:8.12.0
#     environment:
#       - discovery.type=single-node
#       - xpack.security.enabled=false
#
#   kibana:
#     image: kibana:8.12.0
#     ports: ["5601:5601"]

# Kubernetes — Worker Deployment with HPA
# apiVersion: apps/v1
# kind: Deployment
# metadata:
#   name: nuclei-worker
# spec:
#   replicas: 5
#   selector:
#     matchLabels: { app: nuclei-worker }
#   template:
#     spec:
#       containers:
#         - name: worker
#           image: nuclei-worker:latest
#           resources:
#             requests: { cpu: "500m", memory: "256Mi" }
#             limits: { cpu: "1000m", memory: "512Mi" }
# ---
# apiVersion: autoscaling/v2
# kind: HorizontalPodAutoscaler
# metadata:
#   name: nuclei-worker-hpa
# spec:
#   scaleTargetRef:
#     apiVersion: apps/v1
#     kind: Deployment
#     name: nuclei-worker
#   minReplicas: 2
#   maxReplicas: 20
#   metrics:
#     - type: External
#       external:
#         metric: { name: redis_queue_length }
#         target: { type: AverageValue, averageValue: "10" }

dashboard = {
    "Total Targets": "1,500",
    "Scanned": "1,234 (82%)",
    "Remaining": "266",
    "Active Workers": "5/5",
    "Queue Size": "45 jobs",
    "Total Findings": "89",
    "Critical": "3",
    "High": "12",
    "Medium": "38",
    "Low": "36",
    "Scan Speed": "~50 targets/hour",
    "ETA": "5.3 hours",
}

print("\nScan Dashboard:")
for k, v in dashboard.items():
    print(f"  {k}: {v}")

CI/CD Integration

# === CI/CD Pipeline ===

# GitLab CI
# nuclei_scan:
#   stage: security
#   image: projectdiscovery/nuclei:latest
#   script:
#     - nuclei -u $APP_URL
#       -t cves/ -t exposures/ -t misconfigurations/
#       -severity critical, high
#       -json -o nuclei-results.json
#       -stats -silent
#     - |
#       CRITICAL=$(grep -c '"severity":"critical"' nuclei-results.json || true)
#       HIGH=$(grep -c '"severity":"high"' nuclei-results.json || true)
#       if [ "$CRITICAL" -gt 0 ]; then
#         echo "CRITICAL vulnerabilities found! Failing pipeline."
#         exit 1
#       fi
#   artifacts:
#     paths: [nuclei-results.json]
#   allow_failure: false

# GitHub Actions
# - name: Nuclei Scan
#   uses: projectdiscovery/nuclei-action@main
#   with:
#     target: }
#     templates: cves/, exposures/
#     severity: critical, high
#     output: nuclei-results.json
#     github-report: true

# Custom Templates
# id: custom-api-check
# info:
#   name: Custom API Health Check
#   severity: info
# http:
#   - method: GET
#     path: ["{{BaseURL}}/api/health"]
#     matchers:
#       - type: status
#         status: [200]
#       - type: word
#         words: ["healthy"]

scan_results = {
    "CVE-2024-1234": {"severity": "critical", "target": "api.example.com", "type": "RCE"},
    "CVE-2024-5678": {"severity": "high", "target": "web.example.com", "type": "SQLi"},
    "exposed-git": {"severity": "high", "target": "staging.example.com", "type": "Exposure"},
    "default-creds": {"severity": "medium", "target": "admin.example.com", "type": "Misconfig"},
}

print("\nScan Findings:")
for vuln_id, info in scan_results.items():
    print(f"  [{info['severity'].upper()}] {vuln_id}")
    print(f"    Target: {info['target']} | Type: {info['type']}")

เคล็ดลับ

• Templates: อัพเดท Templates ทุกวัน nuclei -update-templates
• Rate Limit: ตั้ง Rate Limit ป้องกัน Target ล่ม
• Custom: เขียน Custom Template สำหรับ App เฉพาะ
• Queue: ใช้ Redis Queue กระจายงาน Retry อัตโนมัติ
• HPA: Scale Workers ตาม Queue Size ไม่ต้อง Manual

Nuclei Scanner คืออะไร

Open Source Vulnerability Scanner Template-based CVE Misconfiguration HTTP DNS TCP SSL 8000+ Templates Community CI/CD Custom Template

สรุป

Nuclei Scanner HA Distributed Queue Redis Workers Docker Kubernetes HPA CI/CD Template CVE Vulnerability Scanning DevSecOps Elasticsearch Dashboard Auto-scale