Deploying Text Generation WebUI on a GPU VPS

In short. Text Generation WebUI is an open-source, browser-based interface for running large language models on hardware you control. On a GPU VPS, it gives you a private chat front end and an API backed by a dedicated NVIDIA GPU. You clone the repository, run the start script, load a model, and open the interface through an SSH tunnel. It suits developers and small teams who want self-hosted LLMs.

Starting a local language model takes a few minutes. Living with one from a terminal gets old quickly, because every change to a prompt template or a setting means editing flags and restarting the process. Text Generation WebUI puts a browser interface in front of the model, so you load a model and adjust it from a tab.

This guide covers what the tool offers and how to set it up on a GPU VPS, then describes the kind of user this setup suits. One naming note first: the project, created by the developer known as oobabooga, is now branded TextGen, and the old GitHub address redirects to the oobabooga/textgen repository. Older tutorials use the earlier name and commands, so check the repository README if a step here differs from what you see on screen.

What does Text Generation WebUI offer?

Text Generation WebUI is a Gradio-based web interface for running large language models (LLMs) locally, with chat, free-form text generation, and an OpenAI-compatible API in a single application. It runs without telemetry or outside calls, so your prompts and documents stay on the server you deploy it to.

The interface is organized into tabs. The Chat tab handles conversations, including custom characters and instruction-following templates. The Notebook tab lets the model continue any text without chat turns, which helps with drafting and prompt testing. Sampling parameters and generation options sit in the interface, so you change how the model behaves without touching a config file. You can also attach text files, PDFs, and Word documents and ask the model about their contents.

What separates it from a plain API server is that the interface and the API share one loaded model. A bare inference server answers requests and does nothing else. Here, you test a prompt in the browser, then point your own application at the same instance once the output looks right. The API exposes OpenAI-compatible Chat and Completions endpoints, including tool calling, so client code written for the OpenAI API usually needs only a changed base URL.

Extensibility is the third difference. Extensions add features such as text-to-speech, voice input, and translation, and the community maintains many more. The tool also loads models through several backends, including llama.cpp, Transformers, and ExLlamaV3. That choice matters on a GPU server, because the model format and quantization decide how much VRAM a model needs and how fast it responds.

How do you set up Text Generation WebUI on a GPU VPS?

You set up Text Generation WebUI on a GPU VPS in five steps: check the GPU, clone the repository and run the start script, load a model, open the interface through an SSH tunnel, and optionally enable the API. A Contabo GPU VPS ships on Ubuntu 24.04 with CUDA and nvidia-container-toolkit already installed, so you skip the driver and CUDA installation that GPU servers usually need first.

1. Connect to your GPU VPS over SSH and confirm the GPU is visible. Running nvidia-smi should list the NVIDIA card and its VRAM.
2. Clone the repository and run the start script. When it asks which GPU vendor you use, select NVIDIA. The script builds a self-contained Conda environment in a folder called installer_files, so it leaves your system Python alone. The full install needs about 10 GB of disk space.
3. Load a model. In the Model tab, download a model from Hugging Face, select it, choose a loader, and load it. For GGUF models on NVIDIA hardware, the loader places layers on the GPU automatically.
4. Open the interface through an SSH tunnel. The web UI listens on port 7860 and binds to the local machine by default, so you forward that port to your own computer and browse to it there.
5. Optionally enable the API. Add --api to user_data/CMD_FLAGS.txt and restart the server. The API listens on port 5000 by default, separate from the web UI on port 7860, so it needs its own access path. Forward port 5000 through the same SSH tunnel, then send OpenAI-style requests to http://127.0.0.1:5000/v1 from your own computer.

A portable build is an alternative path if you only plan to run GGUF models. You download a release, unzip it, and run it, with no Conda environment to manage. Recent portable builds open a native desktop window, so on a headless server you pass --listen or --nowebui to run the server directly. Other model formats and extensions need the full installation shown here.

On the server, steps 1 and 2 look like this:

nvidia-smi
git clone https://github.com/oobabooga/textgen
cd textgen
./start_linux.sh

On your own computer, steps 4 and 5 look like this. Replace user and YOUR_SERVER_IP with your login and the server address:

ssh -L 7860:127.0.0.1:7860 -L 5000:127.0.0.1:5000 user@YOUR_SERVER_IP

With the tunnel open, go to http://127.0.0.1:7860 in your local browser. The interface loads as if it were running on your own machine, while the model runs on the GPU VPS.

Keeping the server running and up to date

The server stops when your SSH session ends unless you detach it. Start it inside tmux or screen, or wrap the start script in a systemd service if you want it to survive reboots. To update the installation later, run update_wizard_linux.sh from the same folder.

Why a tunnel instead of opening the port

Starting the server with --listen makes it reachable from the network, and recent releases restrict cross-origin (CORS) access to localhost by default for that reason. A tunnel keeps the interface off the public internet entirely, which is the safer default for a tool you do not want exposed without access control. If you need direct access for a team, open port 7860 and, if the API is on, port 5000 with Contabo Firewall in the Customer Control Panel, and allow them only from your own source IPs. For the API, also set a key with --api-key, and if you place a reverse proxy in front, add both ports to its configuration.

How much VRAM do you need?

VRAM decides which models you can load, not just how fast they run. On a 96 GB card, a 70-billion-parameter model at FP8 precision has weights of roughly 70 GB, which leaves headroom for context. Quantizing to 4 bits shrinks the weights further, and a 7-billion-parameter model needs a small fraction of that. Context length also consumes VRAM, because the cache grows with every token in the conversation. If loading fails with an out-of-memory error, lower the context size before you switch to a smaller model. Start with a quantized GGUF model, watch memory use in nvidia-smi, and move up in size from there. The 900 GB NVMe disk on a GPU VPS leaves space for a sizable model library, since a single large model file can run to tens of gigabytes.

Who is this setup a good fit for?

This setup fits developers and small teams who want a private, browser-based LLM they can tune, test, and call from their own code, without sending prompts to a third-party API. Four situations stand out.

Prompt and model evaluation is the first. The interface makes it quick to compare models, loaders, and sampling settings side by side on the same hardware, then move the winning configuration into an application through the API.

Privacy-sensitive prototyping is the second. When documents, source code, or customer text cannot leave infrastructure you control, a self-hosted instance with no telemetry removes one category of risk.

An internal assistant for a small group is the third. A handful of colleagues can share one instance behind a tunnel or a firewall rule, and the extension system covers voice input and translation if the team wants them.

The fourth is a backend for your own application. The OpenAI-compatible API lets an existing client talk to a model you host, which suits development and low-volume production use.

The Contabo GPU VPS pairs a dedicated NVIDIA RTX PRO 6000 with 96 GB of GDDR7 VRAM, 18 vCPUs, 96 GB of RAM, and 900 GB of NVMe storage. The GPU is dedicated to your server rather than a shared or fractional slice. Because the price is flat, a month of experiments costs the same as a month of idle time.

That flat rate is also the main tradeoff. If you run a small model a few hours a week, hourly metering from a pay-as-you-go provider can cost less. A GPU VPS pays off when the server is in regular use. It is also built around a single GPU, so it is the wrong shape for multi-GPU training or for serving many simultaneous users at production scale.

FAQ: Text Generation WebUI