Inference Engine Collection

◆

◆

◆

◆

🔮Psychic

★★

“FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.”

◆

◆

◆

◆

🔥Fire

★★

“校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step”

◆

◆

◆

◆

🌊Water

★★

“Rule engine implementation in Golang”

◆

◆

◆

◆

📦Normal

★★

“OneDiff: An out-of-the-box acceleration library for diffusion models.”

◆

◆

◆

◆

🔥Fire

★★

“Large-scale LLM inference engine”

◆

◆

◆

◆

🔥Fire

★★

“A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.”

◆

◆

◆

◆

🔥Fire

★★

“FeatherCNN is a high performance inference engine for convolutional neural networks.”

◆

◆

◆

◆

🔮Psychic

★★

“Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.”

◆

◆

◆

◆

⚡Electric

★★

“Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.”

◆

◆

◆

◆

🔮Psychic

★★

“Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond”

◆

◆

◆

◆

⚙️Steel

★★

“NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device. ”

◆

◆

◆

◆

🔥Fire

★★

“A highly optimized LLM inference acceleration engine for Llama and its variants.”

◆

◆

◆

◆

🔮Psychic

★★

“Python Computer Vision & Video Analytics Framework With Batteries Included”

◆

◆

◆

◆

📦Normal

★★

“MLX Studio - Home of JANG_Q - Image Gen/Edit + Chat/Code All in one - + OpenClaw (Anthropic API)”

◆

◆

◆

◆

🔥Fire

★★

“Adlik: Toolkit for Accelerating Deep Learning Inference”

◆

◆

◆

◆

🔮Psychic

★★

“2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.”

◆

◆

◆

◆

🔥Fire

★★

“🔥 (yolov3 yolov4 yolov5 unet ...)A mini pytorch inference framework which inspired from darknet.”

◆

◆

◆

◆

🔥Fire

★★

“Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O”