왜 이 20개인가
Ranked by GitHub stars: how many developers bookmarked the project. Stars measure attention, not quality, so each row also shows forks, open issues, licence and the last push.
-
1
nomic-ai/gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
77,351 stars 8,265 forks775 open issuesC++last push 2025-05-27MIT
-
2
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
44,076 stars 8,174 forks3,605 open issuesPythonlast push 2026-10-19Apache-2.0
-
3
gitleaks/gitleaks
Find secrets with Gitleaks 🔑
30,131 stars 2,411 forks505 open issuesGolast push 2026-09-30MIT
-
4
liguodongiot/llm-action
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
25,186 stars 2,978 forks20 open issuesHTMLlast push 2026-10-14Apache-2.0
-
5
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
13,718 stars 1,664 forks315 open issuesPythonlast push 2026-10-13Apache-2.0
-
6
bentoml/OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
12,560 stars 999 forks21 open issuesPythonlast push 2026-10-19Apache-2.0
-
7
openvinotoolkit/openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
11,026 stars 3,472 forks876 open issuesC++last push 2026-10-19Apache-2.0
-
8
Tiiny-AI/PowerInfer
High-speed Large Language Model Serving for Local Deployment
9,839 stars 630 forks133 open issuesC++last push 2026-05-11MIT
-
9
FareedKhan-dev/kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
9,110 stars 1,486 forks4 open issuesClast push 2026-10-02Apache-2.0
-
10
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
8,909 stars 1,046 forks235 open issuesPythonlast push 2026-10-05Apache-2.0
-
11
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
8,332 stars 1,727 forks2,064 open issuesRustlast push 2026-10-19NOASSERTION
-
12
InternLM/lmdeploy
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
8,108 stars 787 forks614 open issuesPythonlast push 2026-10-19Apache-2.0
-
13
algorithmicsuperintelligence/openevolve
Open-source implementation of AlphaEvolve
7,600 stars 1,190 forks96 open issuesPythonlast push 2026-10-19Apache-2.0
-
14
katanemo/plano
Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay...
7,095 stars 488 forks147 open issuesRustlast push 2026-10-07Apache-2.0
-
15
drumih/turbo-fieldfare
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
6,944 stars 467 forks53 open issuesSwiftlast push 2026-10-15Apache-2.0
-
16
bojieli/ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
6,733 stars 522 forks8 open issuesPythonlast push 2026-10-01Apache-2.0
-
17
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
6,708 stars 1,615 forks1,270 open issuesCudalast push 2026-10-19Apache-2.0
-
18
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
6,133 stars 1,741 forks208 open issuesGolast push 2026-10-19Apache-2.0
-
19
cactus-compute/cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
6,131 stars 518 forks115 open issuesC++last push 2026-09-26NOASSERTION
-
20
Michael-A-Kuykendall/shimmy
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
5,986 stars 587 forks10 open issuesRustlast push 2026-10-19Apache-2.0
측정 방법
GitHub repository search `topic:llm-inference`, sorted by stars; archived repositories and forks are left out.
참고 자료
- GitHub search: topic:llm-inference, GitHub (GitHub API terms). Checked 2026-10-19 21:49 UTC.
다른 리스트
사진: Nvidia GeForce RTX 5060 Ti 16GB, PNY Overclocked Dual Fan, front by FreeMediaKid!, CC BY-SA 4.0 via Wikimedia Commons. Pictures in rows: Wikimedia Commons, GitHub and the linked pages.