Semgrep SAST Testing Strategy QA

Semgrep SAST Testing

Semgrep SAST ช่วย QA ตรวจ Security Bugs ตั้งแต่ Code Review Custom Rules Anti-patterns CI/CD Pipeline ร่วม Unit Tests Integration Tests Quality Gate

Testing Strategy 4 ระดับ Testing Pyramid Unit Integration E2E Manual Semgrep Static Analysis ก่อน Unit Tests

Testing Pyramid กับ Security

# testing_pyramid.py — Testing Pyramid with Security
from dataclasses import dataclass, field
from typing import List, Dict

@dataclass
class TestLevel:
    name: str
    count: str
    speed: str
    scope: str
    tools: List[str]
    security_tools: List[str]

class TestingPyramid:
    """Testing Pyramid with Security Layers"""

    def __init__(self):
        self.levels: List[TestLevel] = []

    def add(self, level: TestLevel):
        self.levels.append(level)

    def show(self):
        print(f"\n{'='*55}")
        print(f"Testing Pyramid + Security")
        print(f"{'='*55}")

        for level in self.levels:
            print(f"\n  [{level.name}]")
            print(f"    Count: {level.count} | Speed: {level.speed}")
            print(f"    Scope: {level.scope}")
            print(f"    Tools: {', '.join(level.tools)}")
            print(f"    Security: {', '.join(level.security_tools)}")

pyramid = TestingPyramid()

levels = [
    TestLevel("Static Analysis (SAST)", "ทุก Commit", "เร็วมาก (วินาที)",
              "โค้ดทั้งหมด ไม่ต้องรัน",
              ["Semgrep", "ESLint", "Pylint"],
              ["Semgrep Security Rules", "gitleaks", "Checkov"]),
    TestLevel("Unit Tests", "มากที่สุด (70%)", "เร็ว (วินาที)",
              "ฟังก์ชันเดียว แยกส่วน",
              ["pytest", "Jest", "JUnit"],
              ["Security Unit Tests", "Input Validation Tests"]),
    TestLevel("Integration Tests", "ปานกลาง (20%)", "ปานกลาง (นาที)",
              "หลายส่วนทำงานร่วมกัน",
              ["pytest", "Testcontainers", "Supertest"],
              ["Auth Integration Tests", "API Security Tests"]),
    TestLevel("E2E Tests", "น้อย (10%)", "ช้า (นาที-ชั่วโมง)",
              "ระบบทั้งหมดจากมุม User",
              ["Playwright", "Cypress", "Selenium"],
              ["OWASP ZAP", "Nuclei", "Security E2E"]),
    TestLevel("Manual/Exploratory", "น้อยที่สุด", "ช้ามาก",
              "หา Edge Cases ที่ Automated ไม่เจอ",
              ["Manual Testing", "Bug Bounty"],
              ["Penetration Testing", "Security Audit"]),
]

for level in levels:
    pyramid.add(level)

pyramid.show()

# QA Metrics
qa_metrics = {
    "Code Coverage": {"target": "> 80%", "tool": "pytest-cov, Istanbul"},
    "Semgrep Findings (ERROR)": {"target": "0", "tool": "Semgrep"},
    "Semgrep Findings (WARNING)": {"target": "< 5", "tool": "Semgrep"},
    "Unit Test Pass Rate": {"target": "100%", "tool": "pytest, Jest"},
    "Integration Test Pass Rate": {"target": "100%", "tool": "Testcontainers"},
    "E2E Test Pass Rate": {"target": "> 95%", "tool": "Playwright"},
    "DAST Findings (Critical)": {"target": "0", "tool": "OWASP ZAP"},
    "Mean Time to Fix (Security)": {"target": "< 48 ชั่วโมง", "tool": "Jira"},
}

print(f"\n\nQA Metrics:")
for metric, info in qa_metrics.items():
    print(f"  {metric}: Target {info['target']} ({info['tool']}")

Semgrep Custom Rules สำหรับ QA

# semgrep_qa_rules.py — Custom Rules for QA

# rules:
#   # Rule 1: ห้ามใช้ print() ใน Production Code
#   - id: no-print-in-production
#     patterns:
#       - pattern: print(...)
#       - pattern-not-inside: |
#           def test_...(...)
#     paths:
#       exclude:
#         - tests/
#         - scripts/
#     message: "Remove print() from production code. Use logging instead."
#     severity: WARNING
#     languages: [python]
#
#   # Rule 2: ต้องมี Error Handling
#   - id: missing-error-handling
#     patterns:
#       - pattern: |
#           requests.get(...)
#       - pattern-not-inside: |
#           try:
#             ...
#     message: "HTTP requests must be wrapped in try/except."
#     severity: WARNING
#     languages: [python]
#
#   # Rule 3: ห้าม Hardcode URLs
#   - id: hardcoded-url
#     pattern: |
#       $VAR = "https://..."
#     message: "Use config or environment variable for URLs."
#     severity: INFO
#     languages: [python]
#
#   # Rule 4: ต้องใช้ Parameterized Queries
#   - id: sql-string-concat
#     patterns:
#       - pattern: |
#           $CURSOR.execute("..." + $VAR + "...")
#     message: "Use parameterized queries to prevent SQL injection."
#     severity: ERROR
#     languages: [python]
#
#   # Rule 5: Test Functions ต้องมี Assert
#   - id: test-without-assert
#     patterns:
#       - pattern: |
#           def test_$NAME(...):
#             ...
#       - pattern-not: |
#           def test_$NAME(...):
#             ...
#             assert ...
#     message: "Test function must contain at least one assert."
#     severity: WARNING
#     languages: [python]

qa_rules = {
    "Security": [
        "no-hardcoded-secrets: ห้าม Hardcode Passwords/API Keys",
        "sql-injection: ต้องใช้ Parameterized Queries",
        "missing-auth: API Endpoints ต้องมี Authentication",
        "insecure-random: ใช้ secrets.token_hex() แทน random()",
    ],
    "Code Quality": [
        "no-print: ห้าม print() ใน Production Code",
        "error-handling: HTTP Requests ต้องมี try/except",
        "hardcoded-url: ห้าม Hardcode URLs",
        "max-complexity: Cognitive Complexity < 15",
    ],
    "Testing": [
        "test-assert: Test Functions ต้องมี Assert",
        "mock-external: ต้อง Mock External Services",
        "test-naming: Test Names ต้องอธิบายสิ่งที่ทดสอบ",
        "no-sleep: ห้ามใช้ time.sleep() ใน Tests",
    ],
}

print("Semgrep Custom Rules for QA:")
for category, rules in qa_rules.items():
    print(f"\n  [{category}]")
    for rule in rules:
        print(f"    - {rule}")

CI/CD Quality Pipeline

# qa_pipeline.py — QA Pipeline with Semgrep

# GitHub Actions — Full QA Pipeline
# name: QA Pipeline
# on: [push, pull_request]
# jobs:
#   static-analysis:
#     runs-on: ubuntu-latest
#     steps:
#       - uses: actions/checkout@v4
#       - name: Semgrep Security Scan
#         uses: returntocorp/semgrep-action@v1
#         with:
#           config: p/security-audit ./custom-rules/
#       - name: Lint
#         run: |
#           pip install ruff
#           ruff check .
#
#   unit-tests:
#     runs-on: ubuntu-latest
#     needs: static-analysis
#     steps:
#       - uses: actions/checkout@v4
#       - name: Run Unit Tests
#         run: |
#           pip install -r requirements.txt
#           pytest tests/unit/ -v --cov=src --cov-report=xml
#       - name: Upload Coverage
#         uses: codecov/codecov-action@v3
#
#   integration-tests:
#     runs-on: ubuntu-latest
#     needs: unit-tests
#     services:
#       postgres:
#         image: postgres:15
#         env:
#           POSTGRES_PASSWORD: test
#     steps:
#       - uses: actions/checkout@v4
#       - name: Run Integration Tests
#         run: pytest tests/integration/ -v
#
#   e2e-tests:
#     runs-on: ubuntu-latest
#     needs: integration-tests
#     steps:
#       - uses: actions/checkout@v4
#       - name: Playwright E2E Tests
#         run: |
#           npx playwright install
#           npx playwright test
#
#   security-gate:
#     runs-on: ubuntu-latest
#     needs: [static-analysis, unit-tests, integration-tests]
#     steps:
#       - name: Check Quality Gate
#         run: |
#           echo "All checks passed - Quality Gate OK"

pipeline_stages = {
    "1. Static Analysis": {
        "tools": "Semgrep, Ruff, ESLint",
        "gate": "0 ERROR findings",
        "duration": "30 วินาที",
    },
    "2. Unit Tests": {
        "tools": "pytest, Jest",
        "gate": "100% pass, Coverage > 80%",
        "duration": "1-3 นาที",
    },
    "3. Integration Tests": {
        "tools": "pytest + Testcontainers",
        "gate": "100% pass",
        "duration": "3-10 นาที",
    },
    "4. E2E Tests": {
        "tools": "Playwright, Cypress",
        "gate": "> 95% pass",
        "duration": "5-15 นาที",
    },
    "5. Security Gate": {
        "tools": "Semgrep + OWASP ZAP",
        "gate": "0 Critical/High findings",
        "duration": "1 นาที",
    },
    "6. Deploy": {
        "tools": "ArgoCD, Flux",
        "gate": "All gates passed",
        "duration": "2-5 นาที",
    },
}

print("QA Pipeline Stages:")
for stage, info in pipeline_stages.items():
    print(f"\n  [{stage}]")
    for key, value in info.items():
        print(f"    {key}: {value}")

Best Practices

• Shift Left: ใช้ Semgrep ตั้งแต่ Pre-commit ไม่ต้องรอ CI/CD
• Custom Rules: เขียน Rules เฉพาะทีม ตรวจ Anti-patterns ที่เคยพบ
• Quality Gate: กำหนด Gate ชัดเจน ERROR = 0 ถึง Deploy ได้
• Coverage: ตั้งเป้า 80%+ แต่โฟกัส Critical Paths มากกว่าตัวเลข
• SAST + DAST: ใช้ทั้ง SAST (Semgrep) และ DAST (ZAP) ร่วมกัน
• Fix SLA: Critical Findings แก้ภายใน 24 ชั่วโมง

Semgrep ใช้กับ QA Testing อย่างไร

ตรวจ Security Bugs ตั้งแต่ Code Review Custom Rules Anti-patterns CI/CD Pipeline ร่วม Unit Tests Integration Tests Quality Gate ไม่ให้โค้ดมีช่องโหว่ผ่าน

สรุป

Semgrep SAST ร่วม QA Testing Strategy Testing Pyramid Static Analysis Unit Integration E2E Custom Rules Quality Gate Security Pipeline SAST DAST Coverage Fix SLA Pre-commit CI/CD