termux-llamacpp
v1.5.0
Published
GGUF runtime, model manager & OpenAI server utilizing device resources for Android Termux & ARM64
Maintainers
Keywords
Readme
Termux-LlamaCpp (Python)
Production-Grade GGUF LLM Runtime Utilizing Device Resources, Model Manager & OpenAI Server for Android ARM64
Installation
pip install termux-llamacppQuickstart
from termux_llamacpp import LlamaRuntime, RuntimeConfig
# 1. Initialize Runtime with Vulkan GPU Acceleration
config = RuntimeConfig(
model_path="models/Llama-3.2-1B-Instruct-Q4_K_M.gguf",
device="auto",
threads=4,
context_size=2048
)
runtime = LlamaRuntime(config)
# 2. Synchronous or Streaming Text Generation
output = runtime.generate("한국의 사계절 중 가을의 매력에 대해 설명해줘.", max_tokens=256)
print(output.text)
print(f"Speed: {output.metrics.eval_tokens_per_sec:.2f} t/s (Prompt: {output.metrics.prompt_tokens_per_sec:.2f} t/s)")Description
Ships verified Android ARM64 native binaries with bundled shared libraries and device resource optimization, enabling instant zero-compilation local inference and a robust OpenAI-compatible REST/SSE server.
Documentation
License
Apache-2.0 License. Copyright (c) 2026 Eunho Kim (@uno-km).
