In short. A llamafile is a single executable that bundles an LLM’s weights together with the llama.cpp runtime, so one file runs on Linux, macOS, Windows, and BSD without an install step. On a VPS you download it, run chmod +x, and start it. It serves a chat UI and an OpenAI-compatible API on port 8080. GPU offloading is optional, through -ngl 999.
Most self-hosting routes ask you to install a runtime, fetch weights separately, then reconcile the versions of the two. Llamafile removes both of those steps by shipping them already joined. The project started at Mozilla Builders and is now maintained by Mozilla.ai, and it works by combining llama.cpp with Cosmopolitan Libc, a C library that produces one binary able to run on several operating systems and CPU architectures. On a rented server, where every extra dependency is one more thing to patch, that property does real work for you.
What makes llamafile different
Llamafile is different because the model and the program that runs it are the same file. A conventional setup has a runtime on one side, a GGUF weights file on the other, and a version relationship between them that you maintain. A llamafile has a runtime, the weights, and the GPU backend libraries concatenated into one executable that you copy around like any other binary.
The portability comes from Cosmopolitan Libc, which produces an Actually Portable Executable. One build runs on Linux 2.6.18 and later, macOS 23.1.0 and later, Windows 10 and later on AMD64, FreeBSD 13 and later, NetBSD 9.2 and later, and OpenBSD 7.0 to 7.4. On UNIX systems llamafile extracts a small loader called ape into $TMPDIR/.ape-1.10 and uses it to map the model into memory. On Windows the same file runs natively once it has a .exe extension.
That design has one obvious cost: the files are large. The smallest pre-built example the project publishes, a Qwen3.5 0.8B build at Q8_0, is 1.77 GB. Larger models get correspondingly heavier, and Windows caps executables at 4 GB, which is why the project also publishes the runtime on its own. Download llamafile without weights, point it at any GGUF file with -m, and you get the same program with the size problem removed. Two runtime builds exist: llamafile ships with the CUDA and Vulkan backends bundled as x86-64 Linux and Windows artifacts, while llamafile-thin leaves them out for a smaller download and expects a GPU backend library to already be present.
The licensing is permissive. The llamafile project is Apache 2.0, and the changes it carries to llama.cpp and whisper.cpp stay under MIT so they remain upstreamable. Nothing in that stack bills you per token.
What you need before you start
You need a VPS running a 64-bit Linux with root or sudo access over SSH, plus curl and enough free disk to hold the file you are about to download. The hardware floor is low but real: AMD64 processors must support AVX or llamafile prints an error and refuses to run, and ARM64 processors must be ARMv8a or newer. Any Intel Core or later and any AMD K8 or later clears the AVX bar.
Size your RAM to the quantized weights rather than to the parameter count, and give yourself headroom on top of that for the context window. The practical floor is the size of the file you are about to download, which has to fit on disk first and then largely in memory. Decide up front who should be able to reach the server, because the default behavior binds a web UI and an API to port 8080 with no authentication in front of it.
Running a llamafile on your VPS
Four steps take you from an empty server to an answering API endpoint.
1. Download a pre-built llamafile. The project publishes them on Hugging Face. The Qwen3.5 0.8B build is the smallest and the most likely to start cleanly on modest hardware.
curl -LO https://huggingface.co/mozilla-ai/llamafile_0.10/resolve/main/Qwen3.5-0.8B-Q8_0.llamafile2. Make it executable. This is a one-time step per file on Linux, macOS, and BSD.
chmod +x Qwen3.5-0.8B-Q8_0.llamafile3. Run it. With no mode flag, recent llamafiles start in combined mode: a terminal chat session and an HTTP server on port 8080 at the same time.
./Qwen3.5-0.8B-Q8_0.llamafile4. Verify the endpoint. The server exposes an OpenAI-compatible route, so a single curl confirms the model is loaded and answering.
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{"model":"LLaMA_CPP","messages":[{"role":"user","content":"Say hello"}]}'A JSON object with a choices array means the stack is working end to end.
Once that works, three flags shape how the server behaves. --server runs it headless without the terminal chat, which is what you want under systemd. --port moves it off 8080. --host 0.0.0.0 makes it reachable from other machines, and that one deserves care on a public VPS: binding to all interfaces publishes an unauthenticated inference endpoint to the internet. Keep the default localhost binding and reach it through an SSH tunnel, or put a reverse proxy with authentication in front of it and close port 8080 at the firewall. Every Contabo VPS and VDS includes a free Firewall, managed from the Customer Control Panel, which blocks all inbound traffic by default and leaves port 8080 shut until you open it.
If your VPS has an NVIDIA GPU, add -ngl 999 to offload as many layers as fit. For an agent framework, add --jinja and set --ctx-size to a context window your memory can hold.
When a llamafile refuses to start
Three failures account for most bad first runs, and each has a short fix.
run-detectors or WINE errors on Linux. The kernel has a binfmt_misc registration that claims the file before llamafile can. Register the APE format explicitly and the error goes away:
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo chmod +x /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"The process exits immediately with a CPU error. The host does not advertise AVX. This happens on older or heavily virtualized hardware, and there is no flag that works around it. Move the workload to a node with a newer processor.
Inference runs, but on the CPU. Llamafile falls back quietly when a GPU backend fails to load, so slow generation on a GPU host usually means the backend never came up. Force the issue with --gpu nvidia, which turns the silent fallback into a startup error you can read, and check the startup log for the lines about loading the GPU module. On Linux the CUDA SDK needs to be installed and nvcc needs to be on the PATH for a custom build to succeed.
When portability matters more than raw speed
Llamafile wins on operational simplicity and loses on concurrency. One file, no package manager and no Python environment, and the whole thing moves to another host with scp. For a single-user assistant, a batch job that summarizes documents overnight, an air-gapped box, or a demo you need to hand to a colleague who runs a different operating system, that profile is hard to beat.
Dedicated inference servers are built for the other case. vLLM is built around PagedAttention and continuous batching of incoming requests, SGLang around RadixAttention and prefix caching, and both are aimed at high-throughput serving under concurrent load in a way a llamafile is not. Ollama sits closer to llamafile in spirit and adds a model library and a REST API on port 11434. None of these is the wrong choice. They answer a different question. Pick llamafile when the number of simultaneous users is small and the cost of maintaining a serving stack is the thing you actually want to avoid.
The hardware follows from the same logic. Small quantized models on the CPU run acceptably on a general-purpose Contabo VPS, which makes a llamafile a reasonable first LLM workload on a server you already rent. Larger models, or anything where tokens per second matters to a user who is waiting, want GPU offload and the VRAM to match. A Contabo GPU VPS gives each instance a dedicated NVIDIA RTX PRO 6000 with 96 GB of GDDR7 VRAM alongside 18 vCPUs and 96 GB of RAM, and it ships an Ubuntu 24.04 image with CUDA preinstalled, which removes the driver and toolkit setup that GPU offload otherwise asks of you.
FAQ: self-hosting llamafile
Is a llamafile the same as a regular model file?
No. A GGUF file holds only model weights and needs a separate runtime to do anything. A llamafile holds the weights and the llama.cpp runtime in one executable, so it runs on its own. You can also use the llamafile runtime alone and pass it an external GGUF file with the -m flag, which combines the two approaches.
Does a llamafile need a GPU?
No. Llamafile runs on the CPU by default and needs only an AMD64 processor with AVX support or an ARMv8a or newer ARM64 processor. A GPU changes the speed, not the outcome. On NVIDIA and AMD cards you pass -ngl 999 to offload layers, on Apple Metal offload is on by default, and a Vulkan backend covers much of the rest.
Can I run a llamafile without installing anything?
Yes, that is the design goal. You download one file, mark it executable with chmod +x on Linux, macOS, or BSD, and run it. There is no package manager step, no Python environment, and no container runtime. On Windows you add a .exe extension instead of running chmod. Removing it later means deleting the file.
How do I reach a llamafile server from outside the VPS?
By default the server binds to localhost on port 8080, which is the safe setting. The simplest remote access is an SSH tunnel from your workstation. If you need the endpoint reachable directly, pass –host 0.0.0.0, then put authentication in front of it with a reverse proxy and restrict the port at the firewall, because the API itself has no access control.
Is llamafile free to use commercially?
The llamafile project is released under the Apache 2.0 license, and its changes to llama.cpp and whisper.cpp are MIT licensed, so the software carries no usage fees. The model you run has its own separate license, and those vary considerably between model families. Check the license on the specific weights before you ship anything commercial on top of them.
