Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.
-
Updated
Dec 28, 2024 - Cuda
#
inference-acceleration
Here are 10 public repositories matching this topic...
[NeurIPS 2024] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
distributed-computing text-to-image efficient-inference diffusion-models text-to-video inference-acceleration stable-diffusion training-free
-
Updated
Sep 27, 2024 - Python
⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.
-
Updated
Jun 8, 2024 - Python
Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
latte video-generation diffusion-models inference-acceleration open-sora-plan open-sora cogvideox hunyuan-video
-
Updated
Jan 7, 2025 - Python
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
-
Updated
Jul 16, 2024 - Python
DepthStream Accelerator: A TensorRT-optimized monocular depth estimation tool with ROS2 integration for C++. It offers high-speed, accurate depth perception, perfect for real-time applications in robotics, autonomous vehicles, and interactive 3D environments.
computer-vision deep-learning robotics depth-estimation ros2 monocular-depth-estimation inference-acceleration tensorrt-inference vision-tranformer
-
Updated
Dec 7, 2024 - C++
Convert and run scikit-learn MLPs on Rockchip NPU.
-
Updated
Jul 13, 2023 - Python
Code for paper "TLEE: Temporal-wise and Layer-wise Early Exiting Network for Efficient Video Recognition on Edge Devices"
inference dnn deep-learning-algorithms resnet action-recognition content-analysis video-recognition video-analytics frame-analysis early-exiting inference-acceleration dynamic-inference dnn-inference
-
Updated
Jul 8, 2023 - Python
Modified inference engine for quantized convolution using product quantization
quantization product-quantization edge-machine-learning inference-optimization mobile-deep-learning inference-acceleration
-
Updated
Jul 1, 2022 - C++
Code for paper "Deep Reinforcement Learning based Multi-task Automated Channel Pruning for DNNs"
deep-learning impala deep-reinforcement-learning dnn distributed neural-networks reinforcement-learning-algorithms nas multi-task prunning ijcnn inference-acceleration network-architecture-search
-
Updated
Aug 25, 2023 - Python
Improve this page
Add a description, image, and links to the inference-acceleration topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the inference-acceleration topic, visit your repo's landing page and select "manage topics."