High-Performance HTTP Template Parser

A Go module that provides a high-performance parser based on text/template for HTTP requests. This module is designed to handle non-rewindable HTTP input streams efficiently while providing template caching and automatic recompilation on file changes.

Performance: Capable of processing 40,000+ requests per second with template caching, optimized for high-throughput web applications and microservices.

Features

• Re-readable HTTP Requests: Handles non-rewindable HTTP input streams by buffering them in memory
• Template Caching: High-performance template caching with LRU eviction
• File Watching: Automatic template recompilation when template files change
• Flexible Template Loading: Supports file system and memory-based template loaders
• Rich Request Data: Extracts headers, query parameters, form data, and body content
• Custom Function Maps: Support for custom template functions
• Thread-Safe: Concurrent access safe across goroutines
• Benchmarked Performance: Optimized for high-throughput scenarios

Installation

go get github.com/fabricates/parser

Quick Start

package main

import (
    "bytes"
    "fmt"
    "net/http"
    "strings"
    
    "github.com/fabricates/parser"
)

func main() {
    // Simple configuration (uses MemoryLoader by default)
    config := parser.Config{
        MaxCacheSize: 100,
        FuncMap:      parser.DefaultFuncMap(),
    }
    
    // Create parser
    p, err := parser.NewParser(config)
    if err != nil {
        panic(err)
    }
    defer p.Close()
    
    // Add template dynamically
    err = p.UpdateTemplate("greeting", "Hello {{.Request.Method}} from {{.Request.URL.Path}}!")
    if err != nil {
        panic(err)
    }
    
    // Create HTTP request
    req, _ := http.NewRequest("GET", "http://example.com/api/users", nil)
    
    // Parse template
    var output bytes.Buffer
    err = p.Parse("greeting", req, &output)
    if err != nil {
        panic(err)
    }
    
    fmt.Print(output.String()) // Output: Hello GET from /api/users!
}

Alternative: Explicit MemoryLoader

func main() {
    // Create a memory-based template loader explicitly
    loader := parser.NewMemoryLoader()
    loader.AddTemplate("greeting", "Hello {{.Request.Method}} from {{.Request.URL.Path}}!")
    
    // Create parser configuration
    config := parser.Config{
        TemplateLoader: loader,
        MaxCacheSize:   100,
        WatchFiles:     false,
        FuncMap:        parser.DefaultFuncMap(),
    }
    
    // Create parser
    p, err := parser.NewParser(config)
    if err != nil {
        panic(err)
    }
    defer p.Close()
    
    // Create HTTP request
    req, _ := http.NewRequest("GET", "http://example.com/api/users", nil)
    
    // Parse template
    var output bytes.Buffer
    err = p.Parse("greeting", req, &output)
    if err != nil {
        panic(err)
    }
    
    fmt.Print(output.String()) // Output: Hello GET from /api/users!
}

Core Components

Parser Interface

The main interface provides methods for template parsing and management:

type Parser interface {
    Parse(templateName string, request *http.Request, output io.Writer) error
    ParseWith(templateName string, request *http.Request, data interface{}, output io.Writer) error
    UpdateTemplate(name string, content string) error
    GetCacheStats() CacheStats
    Close() error
}

Template Loaders

File System Loader

Load templates from the file system with optional file watching:

loader := parser.NewFileSystemLoader("/path/to/templates", ".tmpl", true)

config := parser.Config{
    TemplateLoader: loader,
    WatchFiles:     true, // Enable automatic reloading
    MaxCacheSize:   50,
}

p, err := parser.NewParser(config)

Memory Loader

For testing or when templates are embedded:

loader := parser.NewMemoryLoader()
loader.AddTemplate("welcome", "Welcome {{.Custom.username}}!")

config := parser.Config{
    TemplateLoader: loader,
    MaxCacheSize:   10,
}

p, err := parser.NewParser(config)

Request Data Structure

Templates have access to structured request data:

type RequestData struct {
    Request *http.Request           // Original HTTP request
    Headers map[string][]string     // HTTP headers
    Query   map[string][]string     // Query parameters
    Form    map[string][]string     // Form data (for POST requests)
    Body    string                  // Request body as string
    Custom  interface{}             // Custom data passed to ParseWith
}

Dynamic Template Updates

You can dynamically add or update templates at runtime using the UpdateTemplate method:

// Add a new template
templateContent := "Hello {{.Request.Method}} from {{.Request.URL.Path}}!"
err := parser.UpdateTemplate("greeting", templateContent)
if err != nil {
    log.Fatalf("Failed to update template: %v", err)
}

// Later, update the same template with new content
newContent := "Updated: {{.Request.Method}} {{.Request.URL.Path}}"
err = parser.UpdateTemplate("greeting", newContent)
if err != nil {
    log.Fatalf("Failed to update template: %v", err)
}

// Use the updated template
var output bytes.Buffer
err = parser.Parse("greeting", request, &output)

Note: The UpdateTemplate method automatically calculates MD5 hashes of template content for change detection and caching optimization. If you call UpdateTemplate with the same content multiple times, the template will only be recompiled when the content actually changes.

Template Examples

Basic Request Information

Method: {{.Request.Method}}
URL: {{.Request.URL.Path}}
User-Agent: {{index .Headers "User-Agent" 0}}

Query Parameters

{{if .Query.name}}
Hello {{index .Query "name" 0}}!
{{end}}

Form Data

{{if .Form.username}}
Username: {{index .Form "username" 0}}
{{end}}

Request Body

{{if .Body}}
Received: {{.Body}}
{{end}}

Custom Data

customData := map[string]interface{}{
    "user_id": 123,
    "role":    "admin",
}

err := parser.ParseWith("template", request, customData, output)

User ID: {{.Custom.user_id}}
Role: {{.Custom.role}}

Built-in Template Functions

The parser includes useful template functions:

String Functions

• upper: Convert string to uppercase
• lower: Convert string to lowercase
• title: Convert string to title case
• trim: Remove leading/trailing whitespace
• hasPrefix: Check if string starts with prefix
• hasSuffix: Check if string ends with suffix
• contains: Check if string contains substring
• replace: Replace all occurrences of old substring with new
• split: Split string by separator into slice
• join: Join slice of strings with separator
• trimPrefix: Remove prefix from start of string
• trimSuffix: Remove suffix from end of string
• repeat: Repeat string n times
• substr: Extract substring (start, length)

Utility Functions

• default: Provide default value for empty/nil values

Request Functions

• header: Get request header value
• query: Get query parameter value
• form: Get form field value

Example usage:

Name: {{.Custom.name | upper | default "Anonymous"}}
Content-Type: {{header .Request "Content-Type"}}
URL: {{if hasPrefix .Request.URL.Path "/api"}}API Call{{end}}
File: {{trimSuffix .Custom.filename ".txt"}}
Tags: {{join .Custom.tags ", "}}

File Watching

When WatchFiles is enabled, the parser automatically detects template file changes and recompiles them:

config := parser.Config{
    TemplateLoader: parser.NewFileSystemLoader("./templates", ".tmpl", true),
    WatchFiles:     true,
    MaxCacheSize:   100,
}

p, err := parser.NewParser(config)
// Templates will be automatically reloaded when files change

Performance Optimizations

Template Caching

Templates are cached after compilation with LRU eviction:

config := parser.Config{
    TemplateLoader: loader,
    MaxCacheSize:   100, // Cache up to 100 templates
}

// Get cache statistics
stats := p.GetCacheStats()
fmt.Printf("Cache: %d/%d, Hits: %d\n", stats.Size, stats.MaxSize, stats.HitCount)

Re-readable Requests

HTTP request bodies are automatically buffered to allow multiple reads:

// The parser handles this automatically
rereadableReq, err := parser.NewRereadableRequest(originalRequest)
rereadableReq.Reset() // Reset body for re-reading

Error Handling

The module defines specific error types:

var (
    ErrTemplateNotFound = errors.New("template not found")
    ErrWatcherClosed    = errors.New("file watcher is closed")
    ErrInvalidConfig    = errors.New("invalid configuration")
    ErrParserClosed     = errors.New("parser is closed")
)

Testing

Run the test suite:

go test -v

Run benchmarks:

go test -bench=.

Example benchmark results:

BenchmarkParserParse-2           193795    6008 ns/op
BenchmarkRequestExtraction-2     314965    3690 ns/op

Examples

See the /examples directory for complete usage examples:

• examples/basic/: Basic usage with different request types
• More examples coming soon!

Configuration

type Config struct {
    TemplateLoader TemplateLoader    // How to load templates (defaults to MemoryLoader if nil)
    WatchFiles     bool              // Enable file watching (FileSystemLoader only)
    MaxCacheSize   int               // Template cache size (0 = unlimited)
    FuncMap        template.FuncMap  // Custom template functions
}

Default Behavior

If no TemplateLoader is specified in the config, the parser will automatically use a MemoryLoader by default. This allows you to create a parser with minimal configuration:

// Simple config with default MemoryLoader
config := parser.Config{
    MaxCacheSize: 100,
}

p, err := parser.NewParser(config)
if err != nil {
    panic(err)
}

// Add templates dynamically using UpdateTemplate
err = p.UpdateTemplate("greeting", "Hello {{.Request.Method}}!")

Performance

The parser is designed for high performance with several optimizations:

Benchmark Results

Performance characteristics on a typical server (Intel Xeon E5-2680 v2 @ 2.80GHz):

Operation	Throughput	Memory per Op	Allocations
Basic Parsing	~48,000 ops/sec	4.9 KB	67 allocs
Request Extraction	~105,000 ops/sec	4.9 KB	41 allocs
Generic String Output	~50,000 ops/sec	4.5 KB	69 allocs
Generic JSON Output	~22,000 ops/sec	6.2 KB	104 allocs
Template Caching	~89,000 ops/sec	3.7 KB	34 allocs
Template Updates	~85,000 ops/sec	3.3 KB	43 allocs
Re-readable Requests	~437,000 ops/sec	1.2 KB	10 allocs
Complex Templates	~8,000 ops/sec	14.3 KB	268 allocs

Cache Performance

Template caching provides significant performance benefits:

Cache Size	Performance	Memory Efficiency
Size 1	~46,000 ops/sec	Most memory efficient
Size 10	~50,000 ops/sec	Balanced
Size 100	~42,000 ops/sec	Best hit rate
Unlimited	~44,000 ops/sec	Highest memory usage

Recommendation: Use cache size 10-50 for most applications.

Body Size Impact

Request body size affects memory usage and performance:

Body Size	Throughput	Memory per Op
Small (100B)	~45,000 ops/sec	5.2 KB
Medium (10KB)	~15,000 ops/sec	60.9 KB
Large (100KB)	~2,000 ops/sec	625.4 KB

Concurrent Performance

The parser maintains good performance under concurrent load:

• Concurrent Parsing: ~40,000 ops/sec with multiple goroutines
• Thread-safe: No performance degradation with concurrent access
• Lock-free reads: Template cache uses efficient concurrent access patterns

Optimization Tips

1. Use appropriate cache size: 10-50 templates for most applications
2. Minimize template complexity: Simpler templates execute faster
3. Reuse parser instances: Creating parsers has overhead
4. Pre-load templates: Use UpdateTemplate to warm the cache
5. Monitor cache hit rates: Use GetCacheStats() to optimize cache size

Thread Safety

All components are designed to be thread-safe and can be used concurrently across multiple goroutines.

Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.