Self-hosted AI image generator: real specs

What a self-hosted AI image generator really needs

A self-hosted AI image generator is one web app plus one model file. The app is ComfyUI, and it runs on any Linux box. The model is what decides your hardware. An SDXL-class checkpoint is a single 6.94 GB file, and it wants to sit in GPU memory. That one fact sets the whole budget, so read the hardware ladder below before you rent anything.

The honest summary: a CPU-only VPS can install and serve the software, and it can generate images at 512x512 with an older Stable Diffusion 1.5 model in a minute or two per image. The same box running SDXL at 1024x1024 takes ten to twenty minutes per image. That is not a broken setup. It is arithmetic, and this guide explains it.

Why the model size decides the machine

Image generation runs a denoising loop. A 30-step render passes the full model over the image 30 times, and each pass reads every weight. SDXL at half precision is about 6.9 GB of weights, so a 30-step render moves roughly 200 GB through memory before you see a picture.

A GPU with 24 GB of video memory (VRAM, the memory soldered next to the graphics chip) reads at hundreds of gigabytes per second, and it holds all 6.9 GB at once. A CPU VPS reads system RAM at tens of gigabytes per second, and it has no matrix hardware for the convolutions, so the same loop runs one to two orders of magnitude slower. This is the same memory-bandwidth argument that governs text models, and it is worth reading alongside when a VPS with a GPU is actually worth the money.

Image generation differs from text generation in one way that matters. A chat model streams tokens, so a slow machine still feels usable because words appear while you read. An image appears only when the last step finishes. Slow means staring at a progress bar.

The hardware ladder, with real numbers

The block below holds typical figures for one SDXL image at 1024x1024, 30 steps, Euler sampler, as of July 2026. Treat them as order of magnitude. Your sampler, step count and resolution move them.

[
  {
    "label": "8 vCPU VPS, no GPU",
    "seconds_per_image": 780
  },
  {
    "label": "8GB VRAM GPU",
    "seconds_per_image": 32
  },
  {
    "label": "12GB VRAM GPU",
    "seconds_per_image": 18
  },
  {
    "label": "24GB VRAM GPU",
    "seconds_per_image": 9
  }
]

The CPU row is 780 seconds, about thirteen minutes. The 24 GB card is 9 seconds. That is the gap you are buying across.

Read the ladder this way. Below 8 GB of VRAM, SDXL still runs, because ComfyUI offloads layers to system RAM automatically and will drive a card with as little as 1 GB. Offloading costs time on every step, so a 6 GB card lands closer to a minute per image than to thirty seconds. At 8 GB the base model fits and the render is comfortable. At 12 GB you can hold a ControlNet or two alongside the checkpoint without offloading. At 24 GB you can run SDXL plus the refiner plus upscaling in one workflow, and you can start training LoRA adapters, which needs far more memory than generating does.

What a CPU VPS can and cannot do

It can do more than people expect, and less than the marketing implies. Be specific about the line.

A CPU VPS can install ComfyUI, serve the web interface, hold your model library, run the queue, and generate images with no GPU present at all. With Stable Diffusion 1.5 at 512x512 and 20 steps, expect roughly 60 to 150 seconds per image on 8 modern vCPUs with 16 GB of RAM. For a batch job that runs overnight, or a low-volume image endpoint behind a queue, that is genuinely fine.

A CPU VPS cannot give you interactive work. Prompt iteration means twenty renders in an hour, and at thirteen minutes each you get four. It also cannot train. LoRA fine-tuning on CPU is measured in days, not hours, so treat it as unavailable.

The memory rule for CPU-only is different from the GPU rule. The weights load into system RAM, so you need the model size plus working space: about 16 GB of RAM for SDXL, and about 8 GB for SD 1.5. A 4 GB box will start ComfyUI and then get killed by the out-of-memory killer partway through the first render, which shows up as the process vanishing with Killed in dmesg and no Python traceback.

Install ComfyUI on a GPU machine

These are the upstream commands. Start from a clean Ubuntu 24.04 install with the NVIDIA driver already present. Confirm the driver first, because every later failure looks the same without this check.

nvidia-smi

That must print a table with your card and a CUDA version. command not found, or NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver, means the driver is missing or the kernel module did not load after an upgrade. Fix that before continuing, because ComfyUI will silently fall back to CPU and you will blame the software.

sudo apt update
sudo apt install -y git python3-venv python3-pip wget
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3 -m venv venv
. venv/bin/activate
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt

The virtual environment is not optional advice. PyTorch pulls a large dependency tree, and installing it system-wide on a box that runs anything else is how you break the other thing. Verify PyTorch can see the card before you go further.

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

True plus your card name means the stack is working. False means the wheel you installed does not match the driver, usually because a CPU-only torch wheel was already cached. Reinstall with the index URL above.

Get a model, and plan for the disk

ComfyUI ships with no weights. Checkpoints go in models/checkpoints, VAE files in models/vae, and LoRA adapters in models/loras.

cd ~/ComfyUI/models/checkpoints
wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

Prefer .safetensors over .ckpt always. A .ckpt file is a pickled Python object, and loading one executes code from whoever built it. The safetensors format holds tensors only, so a hostile file cannot run anything.

Storage is the cost people forget. One SDXL base checkpoint is 6.94 GB. The refiner is another 6 GB. A single ControlNet model runs 1.4 to 2.5 GB, an upscaler 60 to 350 MB, and a LoRA 20 to 400 MB. Anyone who enjoys this downloads a second and third base model within a week. Budget 100 GB of disk for a working install, and watch the outputs directory too: 1024x1024 PNG files are 1 to 2 MB each, and an unattended batch fills a small disk quietly. Put models/ and output/ on a volume you can grow, and back up your workflow JSON files with something like encrypted incremental backups to object storage. The weights are re-downloadable. The workflows you tuned are not.

Run it, and reach it safely

cd ~/ComfyUI
. venv/bin/activate
python main.py --listen 127.0.0.1 --port 8188

ComfyUI serves on port 8188. On a CPU-only box, add --cpu, which forces the CPU path instead of failing on a missing CUDA device.

Bind to 127.0.0.1, not to 0.0.0.0. ComfyUI has no login screen and no user accounts. Anything that reaches the port can queue jobs, read every image you have generated, and install custom nodes, which is arbitrary code execution on your server. Reach it through an SSH tunnel from your laptop instead.

ssh -N -L 8188:127.0.0.1:8188 you@your-server

Then open http://127.0.0.1:8188 locally. If you need real multi-user access, put a reverse proxy with authentication in front of it and keep the app bound to localhost. The same reasoning applies to any unauthenticated self-hosted service, and it is the standard pattern in Docker Compose deployments on a VPS.

Keep it running under systemd

A render queue that dies when your SSH session closes is not a service. Write /etc/systemd/system/comfyui.service.

[Unit]
Description=ComfyUI
After=network-online.target

[Service]
User=comfy
WorkingDirectory=/home/comfy/ComfyUI
ExecStart=/home/comfy/ComfyUI/venv/bin/python main.py --listen 127.0.0.1 --port 8188
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now comfyui
systemctl status comfyui

active (running) and a log line reading To see the GUI go to: http://127.0.0.1:8188 mean it is up. Note that ExecStart names the interpreter inside the virtual environment directly, because systemd does not run your shell profile and activate never happens.

The pragmatic recommendation per budget

If you want to try image generation and cost matters more than speed, take a CPU VPS with 16 GB of RAM, run SD 1.5 at 512x512, and accept a minute or two per image. This is the honest entry point, and it costs a fraction of anything with a GPU attached. It is also the right place to host the model library and the workflows while you decide.

If you iterate on prompts daily, rent a GPU with at least 12 GB of VRAM by the hour from a GPU cloud, and shut it down when you stop. Image generation is bursty work, and an idle GPU billed monthly is the most common way people overspend here. Keep the checkpoints on cheap block storage and mount them.

If you serve other people, or you train LoRA adapters, you need 24 GB of VRAM and a machine you keep. At that point the same box usually earns its keep running a local language model too, which is the setup described in self-hosting an LLM with Ollama.

Whichever rung you pick, measure your own machine before you believe any published figure. Queue the same prompt five times and read the seconds-per-iteration that ComfyUI prints in its console. That number is your real ladder position.

FAQ

Can I run Stable Diffusion without a GPU?

Yes. ComfyUI runs with python main.py --cpu and generates real images with no graphics card present. Expect roughly 60 to 150 seconds per image for Stable Diffusion 1.5 at 512x512, and ten to twenty minutes for SDXL at 1024x1024, on 8 modern vCPUs. That works for overnight batches and low-volume endpoints. It does not work for prompt iteration, and training is out of reach entirely.

How much VRAM do I need for SDXL?

8 GB runs SDXL comfortably at 1024x1024. Below that ComfyUI offloads layers to system RAM automatically and still works, down to about 1 GB of VRAM, but every offloaded step costs time. 12 GB lets you hold a ControlNet alongside the checkpoint, and 24 GB covers base plus refiner plus upscaling in one workflow, and is the practical floor for training LoRA adapters.

How much disk space do the models need?

The SDXL base checkpoint alone is 6.94 GB, and the refiner adds about 6 GB more. ControlNet models are 1.4 to 2.5 GB each, LoRA adapters 20 to 400 MB, and upscalers up to 350 MB. Budget 100 GB for a working install with a few base models, and watch the output directory separately, since 1024x1024 PNG files run 1 to 2 MB each.

Is it safe to expose ComfyUI on the public internet?

No. ComfyUI has no authentication of any kind, and its custom-node system installs and runs Python code from the interface, so an open port is remote code execution on your server. Bind it to 127.0.0.1, reach it over an SSH tunnel with ssh -N -L 8188:127.0.0.1:8188 you@your-server, and put an authenticating reverse proxy in front of it if more than one person needs access.

Why did my render get killed with no error message?

The process disappearing with Killed in dmesg and no Python traceback is the Linux out-of-memory killer, not a ComfyUI bug. On a CPU-only box the weights live in system RAM, so SDXL needs about 16 GB and SD 1.5 about 8 GB, plus working space. Add RAM, add swap, or drop to a smaller model and a lower resolution.