왜 이 20개인가
Ranked by GitHub stars: how many developers bookmarked the project. Stars measure attention, not quality, so each row also shows forks, open issues, licence and the last push.
-
1
nomic-ai/gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
77,350 stars 8,264 forks775 open issuesC++last push 2025-05-27MIT
-
2
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
44,088 stars 8,175 forks3,593 open issuesPythonlast push 2026-10-20Apache-2.0
-
3
gitleaks/gitleaks
Find secrets with Gitleaks 🔑
30,159 stars 2,373 forks508 open issuesGolast push 2026-09-30MIT
-
4
liguodongiot/llm-action
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
25,188 stars 2,953 forks20 open issuesHTMLlast push 2026-10-14Apache-2.0
-
5
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
13,718 stars 1,625 forks315 open issuesPythonlast push 2026-10-13Apache-2.0
-
6
bentoml/OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
12,560 stars 955 forks21 open issuesPythonlast push 2026-10-20Apache-2.0
-
7
openvinotoolkit/openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
11,030 stars 3,471 forks882 open issuesC++last push 2026-10-20Apache-2.0
-
8
Tiiny-AI/PowerInfer
High-speed Large Language Model Serving for Local Deployment
9,842 stars 626 forks133 open issuesC++last push 2026-05-11MIT
-
9
FareedKhan-dev/kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
9,126 stars 1,487 forks4 open issuesClast push 2026-10-02Apache-2.0
-
10
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
8,913 stars 1,047 forks234 open issuesPythonlast push 2026-10-20Apache-2.0
-
11
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
8,344 stars 1,729 forks2,094 open issuesRustlast push 2026-10-20NOASSERTION
-
12
InternLM/lmdeploy
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
8,110 stars 780 forks618 open issuesPythonlast push 2026-10-20Apache-2.0
-
13
algorithmicsuperintelligence/openevolve
Open-source implementation of AlphaEvolve
7,605 stars 1,190 forks96 open issuesPythonlast push 2026-10-19Apache-2.0
-
14
katanemo/plano
Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay...
7,096 stars 488 forks147 open issuesRustlast push 2026-10-07Apache-2.0
-
15
drumih/turbo-fieldfare
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
6,961 stars 462 forks53 open issuesSwiftlast push 2026-10-15Apache-2.0
-
16
bojieli/ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
6,793 stars 529 forks8 open issuesPythonlast push 2026-10-01Apache-2.0
-
17
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
6,725 stars 1,622 forks1,261 open issuesCudalast push 2026-10-20Apache-2.0
-
18
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
6,140 stars 1,743 forks211 open issuesGolast push 2026-10-20Apache-2.0
-
19
cactus-compute/cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
6,137 stars 519 forks115 open issuesC++last push 2026-09-26NOASSERTION
-
20
Michael-A-Kuykendall/shimmy
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
5,988 stars 587 forks10 open issuesRustlast push 2026-10-19Apache-2.0
측정 방법
GitHub repository search `topic:llm-inference`, sorted by stars; archived repositories and forks are left out.
참고 자료
- GitHub search: topic:llm-inference, GitHub (GitHub API terms). Checked 2026-10-20 18:57 UTC.
다른 리스트
사진: Nvidia GeForce RTX 5060 Ti 16GB, PNY Overclocked Dual Fan, front by FreeMediaKid!, CC BY-SA 4.0 via Wikimedia Commons. Pictures in rows: Wikimedia Commons, GitHub and the linked pages.