llama.cpp

A plain C/C++ engine for running language models locally on CPUs and GPUs, with quantization, a command-line tool and an OpenAI-compatible server.

133k stars, 25k forks C++ On GitHub since March 2023

Visit website View on GitHub How to run it

What is llama.cpp?

llama.cpp runs large language models and vision models with minimal setup on ordinary hardware. It has no external dependencies, treats Apple silicon as a first-class target through Metal, and uses AVX, CUDA and other backends elsewhere. Quantization from 1.5 to 8 bits shrinks models so they fit in less memory. It downloads GGUF models directly from Hugging Face, chats with them from the command line, and serves them through an OpenAI-compatible API with a built-in web UI. Many local AI apps use it under the hood.

Good fit if

  • You want the most direct, lowest-level way to run open models on your own machine.
  • You need to run models on modest hardware, laptops or edge devices.
  • You build software and want to embed local inference as a library.

Look elsewhere if

  • You want a simpler model manager with one-command downloads: Ollama.
  • You serve many users at once on GPU servers: vLLM is built for throughput.

Before you adopt it

llama.cpp is MIT-licensed, maintained by ggml-org with dozens of active contributors, and has no paid edition. Prebuilt binaries for Windows, Linux and macOS come with every release, and releases arrive several times a week.

Drafted with AI from the project's README and repository data, then checked against that data on 30 October 2026. Report a correction

llama.cpp health: is it still maintained?

84 of 100, actively developed

  • Activity
    1,793 commits in the last 90 days, the latest on 1 November 2026.
    30 / 30
  • Contributors
    48 people wrote the latest 100 commits; the most active wrote 11% of them.
    20 / 20
  • Releases
    15 releases in the last 12 months; the latest, v0.8.0, on 28 October 2026.
    19 / 20
  • Replies to issues
    Too few recent issues from outside the team to measure, so this part gets a neutral score.
    10 / 20
  • Security practices
    No OpenSSF Scorecard for this repository yet, so this part gets a neutral score.
    5 / 10

Read from GitHub on 1 November 2026. How the score works

llama.cpp license: is it open source?

llama.cpp uses MIT, an open-source license approved by the OSI.

  1. March 2023First published on GitHub.
  2. TodayMIT.

How to run llama.cpp yourself

Docker
No Docker files in the repository
Compose file
None in the repository
Setup effort
Not rated: no compose file to judge from
Kubernetes
No Helm chart in the repository
Desktop and mobile
Builds for Windows, Linux, macOS, including ARM in GitHub releases
Interface languages
English only

Install instructions: llama.app or the README on GitHub.

Alternatives to llama.cpp

  • KoboldCppKoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install. 93 health · AGPL-3.0 · 12.1k stars
  • vLLMA high-throughput and memory-efficient inference and serving engine for LLMs 84 health · Apache-2.0 · 96k stars
  • PrivateGPTComplete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and… 67 health · Apache-2.0 · 57.6k stars
  • AnythingLLMStop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first… 94 health · MIT · 67.8k stars
  • RAGFlowRAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG… 93 health · Apache-2.0 · 93.3k stars

All 36 projects in AI chat & local LLMs

Questions about llama.cpp

Is llama.cpp open source?

Yes. llama.cpp is published under MIT, an OSI-approved open-source license.

Is llama.cpp still maintained?

Its health score is 84 of 100 (actively developed). It had 1,793 commits in the last 90 days, the latest on 1 November 2026, and its latest release came out on 28 October 2026.

How do I run llama.cpp?

You can run it as an app for Windows, Linux, macOS, including ARM. The README on GitHub has the exact steps.

What are the alternatives to llama.cpp?

Close alternatives in the AI chat & local LLMs category include KoboldCpp, vLLM, PrivateGPT.