Midjourney Prompt Service Mesh Setup — ระบบ

Prompt Service Mesh

Midjourney Prompt Service Mesh Microservices Istio mTLS Circuit Breaking Load Balancing Queue Worker Scaling gRPC Event-driven

Service	Function	Protocol	Scale Strategy
API Gateway	Route Request Auth	HTTP/REST	HPA (CPU)
Prompt Service	Generate Prompt	gRPC	HPA (Request Rate)
Template Service	CRUD Template	gRPC	HPA (CPU)
Queue Service	Manage Job Queue	AMQP/Kafka	Fixed (2-3 replicas)
Generation Worker	Call Midjourney API	HTTP	KEDA (Queue Depth)
Rating Service	Rate Image Quality	gRPC	HPA (CPU)

Microservices Architecture

# === Prompt System Microservices ===

from dataclasses import dataclass, field

@dataclass
class MicroService:
    name: str
    responsibility: str
    tech_stack: str
    dependencies: list
    api: str

services = [
    MicroService("api-gateway",
        "รับ Request จาก Client, Auth, Rate Limit, Route",
        "Kong / Envoy / Nginx",
        [],
        "REST: POST /api/generate, GET /api/templates, GET /api/images"),
    MicroService("prompt-service",
        "สร้าง Prompt จาก Template + Subject, Validate Quality",
        "Python FastAPI + gRPC",
        ["template-service"],
        "gRPC: GeneratePrompt(template_id, subject) → Prompt"),
    MicroService("template-service",
        "CRUD Template, Search, Version Control",
        "Python FastAPI + PostgreSQL + Redis Cache",
        [],
        "gRPC: GetTemplate, CreateTemplate, SearchTemplates"),
    MicroService("queue-service",
        "จัดคิว Generation Job, Priority Queue, Status Tracking",
        "RabbitMQ / Kafka + Redis",
        [],
        "AMQP: Publish Job, Consume Job, Get Status"),
    MicroService("generation-worker",
        "ดึง Job จาก Queue, เรียก Midjourney API, Save Result",
        "Python + Discord API / Midjourney API + S3",
        ["queue-service"],
        "Consumer: Process Job → Call API → Save Image → Notify"),
    MicroService("rating-service",
        "ประเมินคุณภาพภาพ, เก็บ Feedback, A/B Test Results",
        "Python FastAPI + PostgreSQL",
        [],
        "gRPC: RateImage(image_id, score, feedback)"),
]

print("=== Microservices ===")
for s in services:
    print(f"\n  [{s.name}]")
    print(f"    Role: {s.responsibility}")
    print(f"    Stack: {s.tech_stack}")
    print(f"    Deps: {s.dependencies}")
    print(f"    API: {s.api}")

Istio Configuration

# === Istio Service Mesh Configuration ===

# PeerAuthentication - mTLS STRICT
# apiVersion: security.istio.io/v1beta1
# kind: PeerAuthentication
# metadata:
#   name: default
#   namespace: prompt-system
# spec:
#   mtls:
#     mode: STRICT

# DestinationRule - Circuit Breaking + Load Balancing
# apiVersion: networking.istio.io/v1beta1
# kind: DestinationRule
# metadata:
#   name: generation-worker
# spec:
#   host: generation-worker
#   trafficPolicy:
#     connectionPool:
#       tcp:
#         maxConnections: 100
#       http:
#         h2UpgradePolicy: UPGRADE
#         maxRequestsPerConnection: 10
#     outlierDetection:
#       consecutive5xxErrors: 3
#       interval: 30s
#       baseEjectionTime: 60s
#       maxEjectionPercent: 50

# VirtualService - Retry + Timeout
# apiVersion: networking.istio.io/v1beta1
# kind: VirtualService
# metadata:
#   name: generation-worker
# spec:
#   hosts:
#     - generation-worker
#   http:
#     - timeout: 120s  # Generation takes time
#       retries:
#         attempts: 3
#         perTryTimeout: 45s
#         retryOn: 5xx, reset, connect-failure

@dataclass
class IstioConfig:
    resource: str
    purpose: str
    key_settings: str
    impact: str

istio_configs = [
    IstioConfig("PeerAuthentication",
        "mTLS เข้ารหัส Service-to-Service Traffic",
        "mode: STRICT ทุก Namespace",
        "ป้องกัน Traffic Sniffing Man-in-the-Middle"),
    IstioConfig("AuthorizationPolicy",
        "จำกัด Service-to-Service Access",
        "prompt-service → template-service เท่านั้น",
        "ป้องกัน Unauthorized Service Access"),
    IstioConfig("DestinationRule (Circuit Breaking)",
        "ป้องกัน Cascading Failure",
        "consecutive5xxErrors: 3, baseEjectionTime: 60s",
        "Worker ที่พังจะถูก Eject ไม่รับ Traffic"),
    IstioConfig("VirtualService (Retry)",
        "Retry อัตโนมัติเมื่อ API Fail",
        "attempts: 3, perTryTimeout: 45s, retryOn: 5xx",
        "ลด Failure Rate เมื่อ Midjourney API ไม่เสถียร"),
    IstioConfig("VirtualService (Timeout)",
        "Timeout สำหรับ Long-running Request",
        "timeout: 120s สำหรับ Generation",
        "ป้องกัน Request ค้าง ใช้ Resource ไม่จำเป็น"),
    IstioConfig("EnvoyFilter (Rate Limit)",
        "จำกัด Request ต่อ User",
        "10 req/min Free, 100 req/min Premium",
        "ป้องกัน API Abuse ลด Cost"),
]

print("=== Istio Configs ===")
for c in istio_configs:
    print(f"  [{c.resource}] {c.purpose}")
    print(f"    Settings: {c.key_settings}")
    print(f"    Impact: {c.impact}")

Scaling & Monitoring

# === Auto-scaling Configuration ===

@dataclass
class ScaleConfig:
    service: str
    scaler: str
    metric: str
    min_max: str
    cost_note: str

scaling = [
    ScaleConfig("prompt-service",
        "HPA (Horizontal Pod Autoscaler)",
        "CPU > 70% หรือ Request Rate > 100/s",
        "Min: 2, Max: 10",
        "ใช้ Resource น้อย Scale เร็ว"),
    ScaleConfig("template-service",
        "HPA",
        "CPU > 70%",
        "Min: 2, Max: 5",
        "Cache ใน Redis ลด Load มาก"),
    ScaleConfig("generation-worker",
        "KEDA (Queue-based)",
        "Queue Depth > 10 Jobs",
        "Min: 1, Max: 20",
        "ใช้ Spot Instance ลดค่าใช้จ่าย 70%"),
    ScaleConfig("rating-service",
        "HPA",
        "CPU > 70%",
        "Min: 1, Max: 3",
        "Load ต่ำ ไม่ค่อยต้อง Scale"),
    ScaleConfig("queue-service",
        "Fixed",
        "-",
        "Fixed: 3 replicas",
        "RabbitMQ Cluster 3 nodes สำหรับ HA"),
]

print("=== Scaling Configs ===")
for s in scaling:
    print(f"  [{s.service}] {s.scaler}")
    print(f"    Metric: {s.metric}")
    print(f"    Replicas: {s.min_max}")
    print(f"    Cost: {s.cost_note}")

เคล็ดลับ

• KEDA: ใช้ KEDA Scale Worker ตาม Queue Depth ไม่ต้อง Guess
• Circuit Breaking: ตั้ง Circuit Breaking ป้องกัน Midjourney API ล่ม
• Cache: Cache Template ใน Redis ลด Load 80%+
• Spot: ใช้ Spot Instance สำหรับ Worker ลดค่าใช้จ่าย 70%
• Priority: ใช้ Priority Queue Premium User ได้ภาพเร็วกว่า

ทำไมต้องใช้ Service Mesh กับ Prompt System

Microservices mTLS Load Balancing Circuit Breaking Retry Rate Limiting Observability Metrics Tracing ป้องกัน Cascading Failure API Abuse

อ่านเพิ่ม: Microservices คืออะไร? สอนออกแบบ Microservices Architecture · อ่านเพิ่ม: Microservices Communication Patterns คืออะไร? Sync vs Async, · อ่านเพิ่ม: Serverless คืออะไร? สอน AWS Lambda, Cloud Functions และ Func