Forge

Forge / Local & Offline

llama.cpp

Open source

C/C++ LLM inference. Use the CLI when you want fewer moving parts than a desktop app.

No cardMM-friendlySelf-hostableAPI
Official siteGitHub

Free tier

MIT. The runtime under LM Studio, Ollama, and most GGUF tools.

Limits

You assemble the stack. No GUI. Wrong quant = garbage answers.

Best for

  • Headless servers
  • Scripts
  • Tight RAM budgets

How to use it well

  1. Build with your BLAS/Metal/CUDA flags. Grab a Q4_K_M GGUF, not Q8, on a laptop.
  2. Run `llama-cli -m model.gguf -p '…' -n 256` and measure tokens/sec before wiring an agent.
  3. Expose OpenAI-compatible HTTP only on localhost.

Myanmar notes

If LM Studio/Ollama already run, you do not need this yet. Reach for it on a headless VPS.