Maximilian-Winter/llama-cpp-agent
#657 of 783 by Stars among GitHub repos
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.
About
Not Longer Maintained. Use instead ToolAgents or other Python agentic frameworks.
The llama-cpp-agent framework is a tool designed to simplify interactions with Large Language Models (LLMs). It provides an interface for chatting with LLMs, executing function calls, generating structured output, performing retrieval augmented generation, and processing text using agentic chains with tools.
The framework uses guided sampling to constrain the model output to the user defined structures. This way also models not fine-tuned to do function calling and JSON output will be able to do it.
The framework is compatible with the llama.cpp server, llama-cpp-python and its server, and with TGI and vllm…
Excerpted from github.com/Maximilian-Winter/llama-cpp-agent
Latest metrics
| Stars | 651 | 2026-07-16 |
|---|---|---|
| Forks | 71 | 2026-07-16 |
| Commits | 542 | 2026-07-16 |
| Releases | 62 | 2026-07-16 |
| Watchers | 14 | 2026-07-16 |
| Open issues | 22 | 2026-07-16 |
| Open PRs | 3 | 2026-07-16 |