Skip to content

Conversation

@christian-byrne
Copy link
Contributor

Add Vite Build System with Content Hashing for Infinite Caching

🎯 Overview

This PR adds a Vite-based build system that enables infinite caching of workflow templates by using content-based hashing in filenames, while requiring zero changes to any consumers (frontend, ComfyUI server, or Cloud server).

Build Time: 0.37s for 403 files
Size Reduction: 1.5% (85.95 MB → 84.69 MB) from JSON minification
Files Changed: Only new files added, no existing files modified
Breaking Changes: None - fully backward compatible


🔥 Problems Solved

1. Cache Management Complexity

Before: Special middleware needed to serve different cache headers for index*.json (5 min TTL) vs other files (1 hour TTL)

// Complex logic required
if (path.match(/index.*\.json/)) {
  res.setHeader('Cache-Control', 'public, max-age=300')  // 5 minutes
} else {
  res.setHeader('Cache-Control', 'public, max-age=3600') // 1 hour
}

After: Single cache rule for all files

// Simple - all files immutable
res.setHeader('Cache-Control', 'public, immutable, max-age=31536000')

2. Performance Issues

Before: Users must re-download templates every 5 minutes (index.json TTL)
After: Files cached infinitely, only new versions trigger downloads

3. PyPI Package Size Limit

Current: 86.3 MB / 100 MB (86% of limit)
After: 84.69 MB / 100 MB (85% of limit) with JSON minification
Future: Could reach ~65 MB with WebP recompression (35% reduction)

4. No Minification

Before: JSON files shipped with whitespace (~38% overhead)
After: JSON minified (10,829 bytes → 6,307 bytes per file on average)


🏗️ Solution Architecture

High-Level Strategy

Hash the JSON, rename the assets to match

  1. Hash workflow JSON files based on their content (SHA256, 8 chars)
  2. Rename ALL associated assets (thumbnails, audio, video) to match the JSON's hash
  3. Update the name field in ALL index*.json files to include the hash

Example Transformation

Before (source: templates/):

templates/
├── 01_qwen_t2i_subgraphed.json
└── 01_qwen_t2i_subgraphed-1.webp

After (output: dist/):

dist/
├── 01_qwen_t2i_subgraphed-214b7748.json      # Hash: 214b7748
└── 01_qwen_t2i_subgraphed-214b7748-1.webp    # Same hash!

Index Update (all index*.json files):

{
  "name": "01_qwen_t2i_subgraphed-214b7748",  // Hash included in name
  "mediaType": "image",
  "mediaSubtype": "webp"
}

Why This Works With No Consumer Changes

Frontend code constructs URLs as:

const workflowUrl = `/templates/${template.name}.json`
const thumbnailUrl = `/templates/${template.name}-1.webp`

Since we update template.name to include the hash, these URLs resolve correctly:

  • /templates/01_qwen_t2i_subgraphed-214b7748.json
  • /templates/01_qwen_t2i_subgraphed-214b7748-1.webp

No code changes needed - just different data in index.json!


📝 Implementation Details

Files Added

  1. vite.config.js - Vite build configuration with 2 custom plugins
  2. scripts/build.js - Build orchestration with size measurement
  3. package.json - NPM dependencies (only vite as devDependency)
  4. package-lock.json - Lockfile
  5. Documentation files (COMPATIBILITY_ANALYSIS.md, IMPLEMENTATION_SUCCESS.md, etc.)

Files Modified

None! All existing files remain untouched. templates/ is the source, dist/ is the output.

Vite Configuration Deep Dive

Plugin 1: hash-templates-and-rename-assets

Step 1: Template Grouping

// Groups files by base name
{
  "01_qwen_t2i_subgraphed": {
    json: "01_qwen_t2i_subgraphed.json",
    assets: [
      "01_qwen_t2i_subgraphed-1.webp",
      "01_qwen_t2i_subgraphed-2.webp"  // If multiple thumbnails
    ]
  }
}

Pattern matching logic:

  • JSON files: basename.json → basename is template name
  • Asset files: basename-N.ext → extract basename via regex /^(.+)-(\d+)$/
  • Special files: Skip .gitignore and index*.json

Step 2: JSON Hashing

// Read JSON
let content = fs.readFileSync(jsonPath)

// Minify BEFORE hashing (important!)
const parsed = JSON.parse(content.toString())
content = Buffer.from(JSON.stringify(parsed))  // No whitespace

// Hash minified content
const hash = crypto.createHash('sha256')
  .update(content)
  .digest('hex')
  .slice(0, 8)  // First 8 chars: 4.3 billion combinations

Step 3: Asset Renaming

// For asset "default-1.webp" with JSON hash "5f65f4da"
const suffix = "-1"  // Extracted from filename
const newName = `default-5f65f4da-1.webp`

Plugin 2: update-index-files

Runs in closeBundle hook (after all files emitted)

Step 1: Build Name Mapping

// Scan dist/ for hashed JSON files
// Extract: "default-5f65f4da.json" → {"default": "default-5f65f4da"}
const nameMapping = {
  "01_qwen_t2i_subgraphed": "01_qwen_t2i_subgraphed-214b7748",
  "02_qwen_Image_edit": "02_qwen_Image_edit-03a3065e",
  // ... all templates
}

Step 2: Update All Index Files

// Find all: index.json, index.ja.json, index.zh.json, etc.
const indexFiles = fs.readdirSync(templatesDir)
  .filter(f => f.startsWith('index') && f.endsWith('.json'))

// For each index file
for (const indexFile of indexFiles) {
  const data = JSON.parse(fs.readFileSync(indexFile))

  // Update name field in each template
  for (const category of data) {
    for (const template of category.templates) {
      if (nameMapping[template.name]) {
        template.name = nameMapping[template.name]
      }
    }
  }

  // Write to dist/
  fs.writeFileSync(`dist/${indexFile}`, JSON.stringify(data, null, 2))
}

Critical: ALL index files must be updated, not just index.json!


⚠️ Edge Cases and Gotchas

1. Assets Renamed Based on JSON Hash, Not Their Own Content

Scenario:

  • Change workflow JSON (add a node)
  • Thumbnail image unchanged

Result:

  • JSON hash changes: default-OLD_HASH.jsondefault-NEW_HASH.json
  • Thumbnail renamed: default-OLD_HASH-1.webpdefault-NEW_HASH-1.webp
  • Thumbnail content identical, but filename changed

Impact:

  • Cache miss for thumbnail even though content unchanged
  • User re-downloads identical thumbnail

Why Acceptable:

  • Workflows and their assets typically update together
  • Single extra download is low-cost
  • Alternative (multiple hashes) doesn't work with index.json structure

Why Not Hash Each Asset Separately:

  • Frontend constructs URL as ${name}-1.webp
  • Can only have ONE name value per template
  • Can't embed multiple hashes (one for JSON, one for each asset)

2. Translation Files Must All Have Matching Names

Critical: The name field MUST be identical across all index*.json files:

// index.json
{"name": "default-5f65f4da", "title": "Getting Started"}

// index.ja.json
{"name": "default-5f65f4da", "title": "はじめに"}  // Same name!

// index.zh.json
{"name": "default-5f65f4da", "title": "入门"}     // Same name!

Why: All language versions reference the same workflow file on disk.

Handled By: The update-index-files plugin updates ALL index files automatically.

Failure Mode: If names don't match, some languages would 404 when loading workflows.

3. Files With Unusual Naming Patterns

Standard Pattern: basename-N.ext where N is a digit

  • default-1.webp
  • default-2.webp

Edge Cases Found (warnings in validation):

  • api_wan_text_to_image .json - trailing space in filename! ⚠️
  • ByteDance-Seedance_00003_.json - trailing underscore ⚠️
  • video_wan2_2_14B_t2v (2).json - parens in filename ⚠️

Handled By: Vite plugin skips files it can't parse and logs a warning.

Action: These files should be renamed in source to follow standard patterns.

4. Build Must Happen Before Publishing

Critical Order:

  1. Developer updates source files in templates/
  2. Run npm run build to generate dist/
  3. Commit both templates/ and dist/ (or just templates/ if dist/ is gitignored)
  4. CI/CD runs build again and publishes dist/ contents

Failure Mode: If you publish templates/ instead of dist/:

  • Files won't have hashes
  • Old cache behavior returns
  • No minification
  • Frontend still works but caching breaks

5. Hash Collisions (Theoretical)

Hash Length: 8 hex characters = 4,294,967,296 possible values

Collision Probability (Birthday paradox):

  • 176 templates: ~0.00000036% chance of collision
  • 10,000 templates: ~0.58% chance of collision

Impact If Collision Occurs:

  • Two different workflows would have same filename
  • Build would emit one, overwrite with the other
  • One template would load the wrong workflow

Mitigation:

  • Could increase hash length to 12 chars (collision virtually impossible)
  • Could add collision detection to build script
  • Current risk is negligible (<0.001% with current template count)

Recommendation: Monitor build output for duplicate filenames, increase hash length if we ever hit 1,000+ templates.

6. Index Schema Validation

Templates must match schema: templates/index.schema.json

Critical Fields:

  • name - String, used to construct URLs
  • mediaType - String, determines file type
  • mediaSubtype - String, file extension (webp, mp3, mp4)

After Hashing: name field will contain hash suffix, but still validates as string.

Validation Script: scripts/validate_templates.py runs automatically, still works after hashing.

7. Git Diff Size

Each build changes:

  • 176 JSON workflow files (different hash in filename)
  • 226 asset files (renamed to match JSON hash)
  • 10 index*.json files (name fields updated)

Total: ~400 files changed per build

Impact:

  • Large diffs in PRs
  • Git history may become large

Mitigation Options:

  1. Add dist/ to .gitignore, only track source templates/
  2. Build in CI/CD, don't commit dist/
  3. Commit dist/ but squash before merge

Recommendation: Option 2 (build in CI) is cleanest.

8. Local Development Workflow

If dist/ is gitignored:

# Clone repo
git clone ...
cd workflow_templates

# Install dependencies
npm ci

# Build templates
npm run build

# Now dist/ exists and can be tested

Important: Document this in README for new contributors!

9. Stale Dist Directory

Scenario:

  • Delete a template from templates/
  • Run npm run build
  • Old template still exists in dist/

Cause: Vite config has emptyOutDir: true, but if it fails...

Current Protection: emptyOutDir: true in vite.config.js

Extra Safety: Could add to build.js:

// Before build
if (fs.existsSync(distDir)) {
  fs.rmSync(distDir, { recursive: true })
}

10. Special Files Handling

Files NOT hashed:

  • index*.json - Updated with hashes but not renamed
  • index.schema.json - Copied as-is
  • .gitignore - Skipped
  • README.md - If it exists, skipped

Why: These are metadata files, not template content.

index.json Naming*: Kept as-is so frontend knows where to start:

// Frontend still fetches
const index = await fetch('/templates/index.json')

If we renamed index.json to index-HASH.json, frontend wouldn't know which file to request!


🧪 Testing Instructions

Prerequisites

cd workflow_templates
npm ci  # Install dependencies

Test 1: Build Success

npm run build

Expected Output:

🔍 Scanning templates directory...
📂 Step 1: Grouping files by template...
   Found 176 template groups
🔐 Step 2: Hashing JSON files and renaming assets...
✅ Processed 403 files
📝 Step 3: Updating all index*.json files...
   Found 10 index files to update
   ✓ index.json: Updated 171 template names
   ✓ index.ja.json: Updated 171 template names
   [... all translations ...]
✅ All index files updated with hashed names

Build should complete in < 1 second

Test 2: File Structure Validation

python3 test_dist_structure.py

Expected Output:

🧪 Testing dist/ structure compatibility...
✅ Successful checks: 339
❌ Errors: 0
🎉 All files found! Frontend would be able to load all templates.

Test 3: Manual Spot Check

# Pick a random template
ls templates/ | grep "^01_qwen"
# Output: 01_qwen_t2i_subgraphed.json
#         01_qwen_t2i_subgraphed-1.webp

# Check it exists in dist with matching hashes
ls dist/ | grep "01_qwen_t2i"
# Output: 01_qwen_t2i_subgraphed-214b7748.json
#         01_qwen_t2i_subgraphed-214b7748-1.webp
# ✅ Same hash in both filenames!

# Verify index.json has correct name
jq '.[] | .templates[]? | select(.name | startswith("01_qwen")) | .name' dist/index.json
# Output: "01_qwen_t2i_subgraphed-214b7748"
# ✅ Matches filename!

# Verify translation file also updated
jq '.[] | .templates[]? | select(.name | startswith("01_qwen")) | .name' dist/index.ja.json
# Output: "01_qwen_t2i_subgraphed-214b7748"
# ✅ Same name across all languages!

Test 4: Verify Minification

# Check original file size
ls -lh templates/01_qwen_t2i_subgraphed.json
# Output: -rw-r--r-- 1 user user 19K Oct 23 21:14 templates/01_qwen_t2i_subgraphed.json

# Check minified file size
ls -lh dist/01_qwen_t2i_subgraphed-214b7748.json
# Output: -rw-r--r-- 1 user user 18K Oct 23 21:14 dist/01_qwen_t2i_subgraphed-214b7748.json

# Smaller = minification working ✅

Test 5: Content Integrity

# Verify JSON is still valid after minification
jq . dist/01_qwen_t2i_subgraphed-214b7748.json > /dev/null && echo "Valid JSON ✅"

# Verify content hasn't changed (just whitespace removed)
diff <(jq -S . templates/01_qwen_t2i_subgraphed.json) \
     <(jq -S . dist/01_qwen_t2i_subgraphed-214b7748.json)
# No output = identical content ✅

Test 6: Hash Consistency

# Build twice, hashes should be identical
npm run build
mv dist dist-build1

npm run build
mv dist dist-build2

diff -r dist-build1 dist-build2
# No differences = deterministic hashing ✅

Test 7: Hash Changes With Content

# Modify a template
echo ' ' >> templates/default.json  # Add whitespace

# Rebuild
npm run build

# Check if hash changed
ls dist/default-*.json
# Output: default-NEW_HASH.json
# Hash changed = content-based hashing working ✅

# Restore original
git checkout templates/default.json
npm run build

🚀 Deployment Checklist

Step 1: Update CI/CD Workflow

File: .github/workflows/publish.yml

Changes Needed:

# BEFORE
- name: Setup package
  run: |
    mkdir -p comfyui_workflow_templates/templates/
    cp -r templates/* comfyui_workflow_templates/templates/

# AFTER
- name: Install build dependencies
  run: npm ci

- name: Build templates
  run: npm run build

- name: Setup package
  run: |
    mkdir -p comfyui_workflow_templates/templates/
    cp -r dist/* comfyui_workflow_templates/templates/  # Changed to dist/

Critical: Use dist/* not templates/*!

Step 2: Test in Staging

  1. Deploy to staging environment
  2. Verify index.json loads: curl https://staging.api/templates/index.json
  3. Verify template loads: curl https://staging.api/templates/default-HASH.json
  4. Verify thumbnail loads: curl https://staging.api/templates/default-HASH-1.webp
  5. Check cache headers: curl -I https://staging.api/templates/default-HASH.json
    • Should see: Cache-Control: public, immutable, max-age=31536000

Step 3: Update Cache Middleware (Optional)

If you have special cache logic, simplify it:

Before:

if (path.match(/^\/templates\/index.*\.json$/)) {
  res.setHeader('Cache-Control', 'public, max-age=300')
} else if (path.startsWith('/templates/')) {
  res.setHeader('Cache-Control', 'public, max-age=3600')
}

After:

if (path.startsWith('/templates/')) {
  res.setHeader('Cache-Control', 'public, immutable, max-age=31536000')
}

Even Better (set at CDN/proxy level):

# In nginx/cloudflare/etc
location /templates/ {
    add_header Cache-Control "public, immutable, max-age=31536000";
}

Step 4: Monitor First Deployment

Metrics to Watch:

  1. 404 errors on /templates/* (should stay at 0)
  2. Frontend error logs (check for template load failures)
  3. Cache hit rate (should increase over time)
  4. Response times (should decrease as cache warms)

Red Flags:

  • Spike in 404s → Names don't match, rollback immediately
  • Template errors in frontend → Index file issue, rollback
  • No cache hits → Headers not set correctly, fix config

Step 5: Gradual Rollout (Recommended)

Option A: Canary Deployment

  1. Deploy to 5% of users
  2. Monitor for 24 hours
  3. Increase to 50%
  4. Monitor for 24 hours
  5. Deploy to 100%

Option B: Shadow Deployment

  1. Deploy new templates alongside old ones
  2. Serve both hashed and non-hashed versions
  3. Monitor access logs (hashed should eventually dominate)
  4. Remove old files after 30 days

Step 6: Verify Cache Behavior

First Load (cold cache):

time curl https://prod.api/templates/default-HASH.json
# Should be ~100-200ms (normal download)

Second Load (warm cache):

time curl https://prod.api/templates/default-HASH.json
# Should be ~10-20ms (cached) or HTTP 304

After Template Update:

# Build with new content
npm run build
# Hash changes: default-HASH1.json → default-HASH2.json

# Old URL still cached
curl https://prod.api/templates/default-HASH1.json  # Fast

# New URL cache miss
curl https://prod.api/templates/default-HASH2.json  # Slow

🔙 Rollback Procedure

If Issues Found in Production

Immediate Rollback (5 minutes):

  1. Revert publish.yml changes:

    # Change back to:
    cp -r templates/* comfyui_workflow_templates/templates/
  2. Trigger new deployment:

    git revert <commit-hash>
    git push
  3. Wait for deployment (usually 5-10 minutes)

  4. Verify rollback:

    curl https://prod.api/templates/index.json | jq '.[] | .templates[0] | .name'
    # Should NOT contain hash: "default" (not "default-HASH")

Why Safe:

  • Original templates/ directory unchanged
  • Reverting one commit restores old behavior
  • No database migrations or data changes
  • Frontend code unchanged (works with both hashed and non-hashed)

Partial Rollback (Serve Both)

If you want to rollback gradually:

  1. Copy both to package:

    # In publish workflow
    cp -r templates/* comfyui_workflow_templates/templates/
    cp -r dist/* comfyui_workflow_templates/templates/
  2. Result: Both versions served

    • Old URLs: /templates/default.json
    • New URLs: /templates/default-HASH.json
  3. Allows:

    • Old frontend keeps working
    • New frontend can use hashed URLs
    • Gradual migration
  4. Cleanup Later:

    # After 30 days, remove non-hashed versions
    find templates/ -name "*.json" ! -name "*-[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9].json" -delete

Rollback Cache Headers

If you updated cache middleware:

Revert to old headers:

// Restore short TTLs
if (path.match(/^\/templates\/index.*\.json$/)) {
  res.setHeader('Cache-Control', 'public, max-age=300')
} else if (path.startsWith('/templates/')) {
  res.setHeader('Cache-Control', 'public, max-age=3600')
}

Impact:

  • Hashed files still work
  • Just won't have infinite caching benefits
  • Safe fallback position

🔮 Future Work (Optional Enhancements)

1. WebP Recompression (High Value)

Goal: Reduce package size by ~28% (another 20 MB saved)

Implementation:

// In vite.config.js, add to asset processing
import sharp from 'sharp'

if (ext === '.webp') {
  const originalSize = assetContent.length
  assetContent = await sharp(assetContent)
    .webp({ quality: 85, effort: 6 })
    .toBuffer()

  const newSize = assetContent.length
  console.log(`Compressed ${file}: ${originalSize}${newSize}`)
}

Expected Results:

  • Current: 76.35 MB of WebP files
  • After: ~55 MB (28% reduction)
  • Total Package: 85 MB → ~65 MB (23% reduction)

Tradeoff: Build time increases to ~2-3 seconds (still acceptable)

2. Increase Hash Length

Current: 8 hex chars = 4.3 billion combinations
Proposed: 12 hex chars = 281 trillion combinations

When: If we ever reach >1,000 templates (collision risk increases)

Change:

// In vite.config.js
const hash = crypto.createHash('sha256')
  .update(content)
  .digest('hex')
  .slice(0, 12)  // Changed from 8 to 12

3. Collision Detection

Add to build.js:

const seenHashes = new Set()

for (const [name, hash] of Object.entries(nameMapping)) {
  if (seenHashes.has(hash)) {
    console.error(`❌ COLLISION: ${name} has duplicate hash ${hash}!`)
    process.exit(1)
  }
  seenHashes.add(hash)
}

4. Incremental Builds

Current: Rebuilds all 403 files every time
Proposed: Only rebuild changed files

Implementation:

// Check if source file modified since last build
const sourceTime = fs.statSync(sourcePath).mtime
const distTime = fs.statSync(distPath).mtime

if (sourceTime <= distTime) {
  console.log(`⏭️  Skipping ${file} (unchanged)`)
  continue
}

Benefit: Faster rebuilds during development (0.37s → 0.1s)

5. Pre-commit Hook

Auto-build before commit:

# .git/hooks/pre-commit
#!/bin/bash
npm run build
git add dist/

Benefit: Never forget to build before committing

Tradeoff: Commits take longer (~1 second)

6. Build Validation

Add to build.js:

// After build, verify all templates accessible
const index = JSON.parse(fs.readFileSync('dist/index.json'))

for (const category of index) {
  for (const template of category.templates) {
    const workflowPath = `dist/${template.name}.json`
    if (!fs.existsSync(workflowPath)) {
      throw new Error(`Missing workflow: ${workflowPath}`)
    }
  }
}

console.log('✅ All templates verified accessible')

7. Bundle Analyzer

See what's taking up space:

npm install --save-dev rollup-plugin-visualizer

# Add to vite.config.js
import { visualizer } from 'rollup-plugin-visualizer'

export default defineConfig({
  plugins: [
    // ... existing plugins
    visualizer({ filename: 'bundle-stats.html' })
  ]
})

Benefit: Identify large files to optimize

8. Git LFS for Media Files

If dist/ is committed: Use Git LFS for large media files

# .gitattributes
dist/*.webp filter=lfs diff=lfs merge=lfs -text
dist/*.mp3 filter=lfs diff=lfs merge=lfs -text
dist/*.mp4 filter=lfs diff=lfs merge=lfs -text

Benefit: Keeps git repo small even with committed dist/

9. CDN Deployment Automation

Deploy dist/ directly to CDN:

# In .github/workflows/publish.yml
- name: Deploy to CDN
  run: |
    aws s3 sync dist/ s3://templates-cdn/ \
      --cache-control "public,immutable,max-age=31536000"

Benefit: Templates served from CDN, not bundled in package

10. Source Maps for Templates

Add metadata about source:

// In dist/default-HASH.json
{
  "_meta": {
    "source": "default.json",
    "hash": "5f65f4da",
    "buildTime": "2025-10-23T21:14:36.464Z"
  },
  // ... rest of workflow
}

Benefit: Debugging, tracing issues back to source file


❓ FAQ

Q: Do I need to change any frontend code?

A: No! The frontend already constructs URLs as ${template.name}.json. We just changed what template.name contains (now includes hash).

Q: What if users have cached old index.json?

A: The old index.json has a 5-minute TTL, so it expires quickly. Once refreshed, they'll get the new index with hashed names.

Q: Can I mix hashed and non-hashed files?

A: Yes! You can serve both. The frontend will use whatever name is in index.json, so both work simultaneously.

Q: What happens to old cached files?

A: They stay cached until TTL expires or user clears cache. They won't be accessed anymore (new index.json points to new hashed names).

Q: How do I test this locally?

A: Run npm run build, then point your local ComfyUI to dist/ instead of templates/. No other changes needed.

Q: What if build fails in CI?

A: CI will exit with error code 1, deployment won't proceed. Same as any build failure.

Q: Do translation files need special handling?

A: No, the build script automatically updates ALL index*.json files (10 languages). Just ensure they all exist in templates/.

Q: Can I still manually edit templates?

A: Yes! Edit files in templates/, then run npm run build. Never edit dist/ directly (it gets overwritten).

Q: How do I add a new template?

A: Same as before:

  1. Add workflow JSON to templates/
  2. Add thumbnail to templates/
  3. Update templates/index.json
  4. Run npm run build
  5. Commit (both templates/ and dist/ if tracking dist/)

Q: What if I delete a template?

A: Delete from templates/, remove from index.json, run npm run build. The build empties dist/ first so deleted templates won't carry over.

Q: How big can hash collisions be?

A: With 8-char hashes and 176 templates, collision probability is 0.00000036%. Practically impossible. Could increase hash length if worried.

Q: Does this work with ComfyUI's template browser?

A: Yes! The browser loads index.json and constructs URLs from the name field. No code changes needed.

Q: Can I preview changes before deploying?

A: Yes:

npm run build
cd dist
python3 -m http.server 8080
# Browse to http://localhost:8080

Q: What if Vite breaks in the future?

A: Vite is stable (v5.x), but if needed:

  • Could rewrite build script in pure Node.js
  • Current Vite config is simple (~200 lines)
  • Only uses Vite's plugin API, not complex features

Q: Do I need Node.js in production?

A: No! Node.js only needed for building. Production just serves static files from dist/.

Q: How often should I rebuild?

A: Only when templates change. Build in CI on every commit, or manually before release.

Q: Can this work with incremental updates?

A: Yes! Only changed templates get new hashes. Unchanged templates keep same hash and filename, so caches remain valid.

Q: What about browser cache limits?

A: Browsers have cache limits (~50-100MB typically), but:

  • Users only cache templates they view
  • Old unused templates get evicted naturally
  • Not an issue in practice

Q: Does this affect ComfyUI Cloud?

A: No changes needed! Cloud serves from same package, just with different cache headers.

Q: Can I A/B test hashed vs non-hashed?

A: Yes! Serve both versions, use different index files:

  • /templates/index.json - Non-hashed names
  • /templates/index-hashed.json - Hashed names
  • Frontend loads appropriate index based on feature flag

Q: What if I want to revert a specific template?

A:

  1. Revert in source: git checkout HEAD~1 templates/old-template.json
  2. Rebuild: npm run build
  3. New hash generated, old content served under new hash

Q: How do I debug mismatched hashes?

A:

# Check what hash the JSON should have
sha256sum templates/default.json | cut -c1-8

# Check what hash it actually has in dist/
ls dist/default-*.json

# If different, JSON content changed between source and dist
# Rebuild fixes it: npm run build

📊 Metrics to Track Post-Deployment

Success Metrics

  1. Cache Hit Rate

    • Before: ~40-60% (short TTLs)
    • After: ~95%+ (infinite caching)
    • Measure: CDN/proxy analytics
  2. Bandwidth Usage

    • Before: X GB/day
    • After: Should decrease by ~40-60%
    • Measure: Server logs, CDN analytics
  3. Template Load Time

    • Before: 100-200ms (network request)
    • After (cached): 10-20ms (from cache)
    • Measure: Frontend performance monitoring
  4. Package Size

    • Before: 86.3 MB
    • After: 84.69 MB (1.5% reduction)
    • Future: ~65 MB with WebP recompression
    • Measure: ls -lh dist/
  5. Build Time

    • Current: 0.37s for 403 files
    • Acceptable: < 5s
    • Measure: CI build logs

Warning Metrics

  1. 404 Rate on /templates/*

    • Should stay at 0%
    • Spike = names don't match
  2. Frontend Errors

    • "Failed to load template" errors
    • Should stay at 0
  3. CI Build Failures

    • Build should succeed 100%
    • Failures = file corruption or code bug

📚 Additional Resources

  • COMPATIBILITY_ANALYSIS.md - Detailed analysis of all affected systems
  • IMPLEMENTATION_SUCCESS.md - Technical implementation details
  • POC_RESULTS.md - Original POC findings
  • POC_SUMMARY.md - POC summary for stakeholders
  • test_dist_structure.py - Automated validation test

✅ Pre-Merge Checklist

  • All tests pass (npm run build && python3 test_dist_structure.py)
  • Build completes in < 1 second
  • dist/ contains all expected files (403 files)
  • All index*.json files updated (10 language files)
  • manifest.json generated with correct mappings
  • Documentation reviewed and complete
  • CI/CD workflow update planned
  • Rollback procedure documented
  • Stakeholders notified of upcoming change

🎓 Learning from This Implementation

Key Insights

  1. Content hashing requires filename changes - There's no way around this fundamental requirement
  2. Zero consumer changes is possible - By updating data (index.json) instead of code
  3. Translation files are first-class - Must update ALL language files, not just index.json
  4. Asset renaming is acceptable - Thumbnail getting new name when JSON changes is low-impact
  5. Build systems are fast - 0.37s for 400+ files proves Vite's performance
  6. Validation is critical - Automated tests catch issues before deployment

What Almost Went Wrong

  1. Almost forgot translation files - Initial implementation only updated index.json!
  2. Almost hashed assets independently - Would have broken URL construction
  3. Almost skipped minification - Vite doesn't minify JSON by default, had to add manually
  4. Almost committed dist/ - Caught early, decided to build in CI instead

What Worked Well

  1. POC first - Tested approach before full implementation
  2. Comprehensive documentation - This PR description captures everything
  3. Automated testing - test_dist_structure.py validates output
  4. Zero breaking changes - Backward compatible = safe to deploy

Ready to merge when you are! All edge cases documented for when you return in a few months. 🚀

- Adds vite.config.js with custom plugins for template processing
- Hash JSON workflows based on content (SHA256, 8 chars)
- Rename assets to match their workflow's hash
- Update all index*.json files with hashed names
- Minify JSON files (42% reduction per file)

Benefits:
- Enables infinite caching (immutable files)
- No consumer changes needed (frontend/servers work as-is)
- 1.5% package size reduction (85.95 MB → 84.69 MB)
- Builds in 0.37s (403 files)
@christian-byrne christian-byrne marked this pull request as draft October 24, 2025 02:20
@socket-security
Copy link

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​vite@​5.4.21961008399100

View full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants