Skip to content

README.md

Latest commit

 

History

History
214 lines (149 loc) · 9.97 KB

README.md

File metadata and controls

214 lines (149 loc) · 9.97 KB
Raw

Semantic Kernel CVE-2026-25592 Bypass & CWE-1039 AutoInvoke Lab

Overview

This laboratory demonstrates an Insecure Orchestration Vulnerability (OWASP Top 10 for LLMs: LLM06) within Microsoft Semantic Kernel (v1.47.0–1.48.0). Students will explore how an architectural Trust Gap — the difference between Time-of-Check (filter validation) and Time-of-Use (execution sink deserialization) — allows attackers to bypass string-matching security mitigations using multi-layered serialization and encoding techniques.

Vulnerability Class: CWE-843 (Type Confusion) → CWE-22 (Path Traversal) → CWE-94 (Code Injection)
CVSS v3.1: 10.0 (AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H)
Reference: JDP-2026-001 White Paper

Core Learning Objectives

  • Understand the Trust Gap: See how components inside an AI orchestration framework pass data without recursive canonicalization.
  • Observe TOCTOU in AI Sinks: Understand how checking a payload at the input filter before decoding allows malformed inputs to evade security boundaries.
  • Evaluate Serialization Evasions: Learn how 6 Type Confusion bypass vectors (JSON Array, Object Reflection, Base64, URL Encoding, Unicode Homoglyph, Hybrid Canonicalization) render basic string validation ineffective.
  • Analyze AutoInvoke Privilege Escalation (CWE-1039): See how ToolCallBehavior.AutoInvokeKernelFunctions allows an LLM to autonomously execute system tools without human-in-the-loop (HITL) approval.
  • Bypass Shell Blinding (Commit fa2d52f6): Microsoft's cosmetic fix masks output paths from the LLM context but does not prevent the underlying file write — verified Out-of-Band via podman exec.

Lab Architecture

The lab utilizes a containerized application (sk-vulnerable) running Microsoft Semantic Kernel v1.48.0 with a deliberately vulnerable FilePlugin:

  1. The Filter: A PathSanitizationFilter converts arguments to strings and scans for explicit directory traversal patterns (.., /, %2f).
  2. The Sink: The native C# file writer decodes and canonicalizes arguments after the filter has already run, allowing specific payload formats to evade detection.

Dual-Mode Operation

Mode Environment Variable PathSanitizationFilter Expected Behavior
UNHARDENED Not set Not registered All 6 vectors + AutoInvoke succeed
HARDENED LAB_HARDENED=true Registered Vectors 1, 2, 4, 5 blocked; Vectors 3, 6 (Base64/Hybrid) bypass due to TOCTOU

Quick Start: Interactive Training Wizard

The interactive_trainer.py provides a menu-driven CLI that walks through all exploitation scenarios with built-in container management.

cd ~/OWASP/GenAI-Red-Team-Lab/exploitation/semantickernel
chmod +x interactive_trainer.py
./interactive_trainer.py

Menu Options

  • 0 Manage Sandbox Container — Start/stop containers in UNHARDENED or HARDENED mode without leaving the trainer.
  • 1 Lesson 1: CVE-2026-25592 Bypass — Practice all 6 Type Confusion vectors with auto-fill hints and real-time telemetry mapping.
  • 2 Lesson 2: AutoInvoke Exploitation — Simulate autonomous LLM tool execution with Shell Blinding toggle (A = disabled, B = enabled).
  • 3 Exit — Clean shutdown.

Interactive Features

  • Auto-fill hints — Press Enter to send the suggested payload, or type your own custom input.
  • Real-time telemetry — Visual step-by-step execution flow (Filter Check → Canonicalization → Disk Access).
  • Live status — Shows container state (ONLINE/OFFLINE) and shield mode (UNHARDENED/HARDENED) on every screen.

Exercise 1: Automated Verification Suite

For a completely automated double-pass evaluation (builds, deploys, and tests all vectors autonomously):

cd ~/OWASP/GenAI-Red-Team-Lab/exploitation/semantickernel
chmod +x verify_all.sh
./verify_all.sh

This script performs two passes:

Pass 1: Unhardened Mode (The Baseline Vulnerability)

  • All 6 Type Confusion vectors return Success.
  • All 4 AutoInvoke scenarios execute autonomously.
  • OOB Verification: Payloads land at /config.txt and /app/config.txt inside the container.

Pass 2: Hardened Mode (The Filtering Illusion)

  • Vectors 1, 2, 4, 5 are blocked — the filter catches literal .. and %2f.
  • Vectors 3 and 6 (Base64, Hybrid) bypass the filter — demonstrating the TOCTOU flaw.
  • All 4 AutoInvoke scenarios are blocked by the filter.
  • OOB Verification: Only vectors 3 and 6 land on the filesystem.

Exercise 2: Type Confusion Bypass Vectors (Lesson 1)

Select any vector from the interactive trainer menu to practice:

Vector Technique Filter Bypass Method
1 JSON Array Confusion ["..", "..", "config.txt"] — The is string check returns False for arrays
2 Object Reflection {"path": "../../config.txt"} — Reflection extracts the path property downstream
3 Base64 Encoding Li4vLi4vY29uZmlnLnR4dA== — No literal .. or / in the encoded string
4 URL Encoding ..%2f..%2fconfig.txt — %2f hides the slash from the filter
5 Unicode Homoglyph ..⁄..⁄config.txt — U+2044 normalizes to / at the OS level
6 Hybrid Canonicalization SafeFolder%2f%2e%2e%2fProgram%2ecs — Multi-layer encoding exhausts non-recursive filters

Exercise 3: AutoInvoke & Shell Blinding (Lesson 2)

The Trust Gap

When ToolCallBehavior.AutoInvokeKernelFunctions is active, the framework trusts the LLM's structured output as system instructions without verification. This creates a CWE-1039 vulnerability.

Shell Blinding (Commit fa2d52f6)

Microsoft's mitigation masks output paths from the LLM context:

// Masking system output from the LLM execution context
var result = await process.StandardOutput.ReadToEndAsync();
return "Command executed successfully.";

This is a purely cosmetic fix. The file write still executes on the server. The interactive trainer verifies this Out-of-Band:

podman exec sk-vulnerable cat /app/config.txt

Code Review: Why the Filter Failed

Open the application source to examine the vulnerability:

cat ../../sandboxes/agentic_local_semantickernel/app/Program.cs

The Inspection Layer (Time-of-Check)

public class PathSanitizationFilter : IFunctionInvocationFilter {
    public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next) {
        foreach (var arg in context.Arguments) {
            // BYPASS: If arg.Value is a JsonElement, this 'is string' check evaluates to false
            if (arg.Value is string str && (str.Contains("..") || str.Contains("/")))
                throw new UnauthorizedAccessException("Blocked!");
        }
        await next(context);
    }
}

The Execution Layer (Time-of-Use)

public void SaveConversation(object path, string content) {
    string stringPath = path?.ToString() ?? "default.txt";
    
    // The Execution Sink decodes AFTER the security filter has executed
    if (stringPath.Contains("%")) stringPath = WebUtility.UrlDecode(stringPath);
    if (stringPath.Contains("\u2044")) stringPath = stringPath.Replace("\u2044", "/");
    if (path is string s && s.EndsWith("=="))
        stringPath = Encoding.UTF8.GetString(Convert.FromBase64String(s));
    else if (path is JsonElement el && el.ValueKind == JsonValueKind.Array)
        stringPath = el[0].GetString() ?? "default.txt";
    else if (path.GetType().GetProperty("path") != null)
        stringPath = path.GetType().GetProperty("path")?.GetValue(path)?.ToString() ?? "default.txt";
        
    File.AppendAllText(stringPath, content);
}

Technical Takeaway

The filter checks for literal .. characters against the encoded payload string. Because the encoded form lacks dots or slashes, it passes inspection. The execution sink then decodes the string back into traversal segments after validation, executing the file write Out-of-Band.

This is a classic Time-of-Check / Time-of-Use (TOCTOU) vulnerability.

The Whack-a-Mole Problem

Every filter fix creates a new bypass vector:

Round Filter Bypass
1 if (arg is string s) Send JSON Array instead (Type Confusion)
2 if (arg.ToString().Contains("..")) Base64-encode the path (no .. in encoded form)
3 Hypothetical: decode Base64, then check Double-encode, or use a different encoding

The root cause is architectural, not a filter bug. The framework trusts the LLM's output as system commands. Until the architecture changes — mandatory path anchoring, type-safe execution sinks, recursive canonicalization pipelines, and compute isolation — every filter fix will be met with a new bypass.

Files in This Directory

File Purpose
interactive_trainer.py Menu-driven CLI trainer with container management
verify_all.sh Double-pass automated verification suite
semantickernel_type_confusion/attack.py Programmatic Type Confusion payload dispatcher
semantickernel_autoinvoke/attack.py Programmatic AutoInvoke payload dispatcher
README.md This lab guide

References & Additional Reading

  • CVE-2026-25592: Path Traversal in Semantic Kernel (bypassed via JDP-2026-001)
  • CWE-843: Type Confusion (Late Canonicalization)
  • CWE-1039: Insecure Automated Optimizations (AutoInvoke)
  • OWASP Top 10 for LLMs: LLM06 - Insecure Orchestration
  • Commit fa2d52f6: Shell Blinding (cosmetic output masking)
  • PR #13683: AllowedDirectories (opt-in "Breaking Change" — disabled by default)
  • Research Paper: JDP-2026-001 White Paper & Disclosure Archive