Machine learning inference#406
Merged
Merged
Conversation
…ls documentation with biological neuron inspiration and activation function theory.
… and fix ONNX Runtime execution provider terminology.
…integration chapters
…AI denoising, adaptive sampling, and advanced ML techniques for desktop applications
…nce documentation by adding missing blank lines after list introductions
…add detailed debugging guidance for Vulkan compute synchronization
…del optimization chapters for ML inference
…bedded devices This commit introduces comprehensive cross-compilation documentation covering sysroot setup, Docker-based toolchains, and CMake configuration for ARM targets. Also adds OpenXR integration chapter demonstrating stereo preprocessing, pose reprojection, and thermal management for 90Hz ML inference in XR headsets. Includes thermal PID control loop diagram.
…d preprocessing Implements complete MNIST digit recognition pipeline using Vulkan compute: - Dense layer, ReLU, and softmax compute shaders with push constants - Full inference class with descriptor sets, pipelines, and buffer management - Preprocessing with bounding box detection, scaling, and center-of-mass normalization - ImGui drawing canvas UI with real-time recognition - Windows setup script for Python environment configuration
Corrects `:pp:` attribute definition syntax across multiple files by changing `::pp:` to `:pp:`. Also fixes xref paths to include proper `ML_Inference/` directory prefixes and corrects image paths and navigation links throughout the documentation.
Includes SVG diagrams for: - Desktop: RT denoising U-Net, adaptive sampling, RND curiosity, RL architecture, validation hierarchy - Embedded: Zero-copy dmabuf pipeline, cross-compilation architecture, XR ML pipeline, thermal PID loop - Advanced: Winograd transform flow, cooperative matrix warp distribution
…p dive, and embedded compiler integration This commit reorganizes the ML inference documentation to introduce a three-path teaching approach (third-party libraries, custom Vulkan, and ML compilers). It adds new chapters on IREE, Apache TVM, Mojo/MAX, and OpenXLA compilers, plus guidance for choosing between them. A new NNEF deep dive chapter and embedded ML compiler integration examples are included. Navigation links are updated throughout, and the "Putting It Together" chapter is renumbered from 06 to 07.
…oncrete deployment guidance Rewrites compiler chapters (IREE, TVM, MAX, OpenXLA) and embedded sections to emphasize: - Boundary contracts between app and runtime rather than kernel extraction - Golden reference validation workflow anchoring correctness before tuning - Concrete decision frameworks for choosing compilers with embedded/desktop examples - Deployment checklists framed as explicit design decisions - NNEF conversion path clarifications with layout/precision considerations - Latency distribution metrics (p50/p95/p99) over single-run means Removes aspirational "extract kernels" paths in favor of embedding compiler runtimes as libraries. Adds troubleshooting sections for layout mismatches, numeric drift bisection, and thermal constraints on embedded devices.
… latency/power profiling tools Implements third ML inference backend (IREE) alongside Vulkan and ONNX, with UI toggle in MNIST demo. Adds NNEF model loader with format auto-detection. Enhances Vulkan preprocessing to handle row stride/padding and optimizes upload path. Patches onnx2tf for newer ONNX versions lacking mapping attribute. Improves roadmap model initialization with gentle spatial smoothing kernels (70% center, 30% neighbors). Extends RT optimization with denoised result upsampling, heatmap-guided adaptive tracing, and cleaned timing breakdowns. Adds latency/power sampling tool with sysfs vitals reader for embedded profiling. Updates image classifier to pass stride parameter.
…oduction" Renames `06_nnef_deep_dive.adoc` to `06_nnef_introduction.adoc` and rewrites content to emphasize practical NNEF usage with the official Khronos C++ parser. Repositions NNEF as a first-class native input path alongside ONNX, clarifies adapter pattern for loading into ComputationGraph, and adds build integration guidance. Updates all xref navigation links throughout the chapter files.
# Conflicts: # antora/modules/ROOT/nav.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.