Which VPS Fit Run Stable Diffusion Without Stress?

Wetin a self-hosted AI image generator really need

A self-hosted AI image generator na one web app plus one model file. The app na ComfyUI, and e fit run for any Linux box. Na the model dey decide the hardware wey you need. An SDXL-class checkpoint na one 6.94 GB file, and e need stay for GPU memory. Na this one fact dey set the whole budget, so read the hardware ladder below before you rent anything.

The honest summary be say: CPU-only VPS fit install and serve the software, and e fit generate images for 512x512 with older Stable Diffusion 1.5 model in one or two minutes per image. The same box wey dey run SDXL for 1024x1024 fit take ten to twenty minutes per image. This no mean say the setup spoil. Na arithmetic, and this guide go explain am.

Why model size dey decide the machine

Image generation dey run a denoising loop. One 30-step render dey pass the full model over the image 30 times, and every pass dey read all the weights. SDXL for half precision na about 6.9 GB of weights, so one 30-step render dey move roughly 200 GB through memory before you see any picture.

GPU wey get 24 GB of video memory (VRAM, the memory wey dem solder next to the graphics chip) dey read at hundreds of gigabytes per second, and e fit hold all 6.9 GB at once. CPU VPS dey read system RAM at tens of gigabytes per second, and e no get matrix hardware for the convolutions, so the same loop dey run one to two orders of magnitude slower. Na this same memory-bandwidth argument dey control text models, and e good make you read am together with when VPS with GPU really worth the money.

Image generation dey different from text generation for one important way. Chat model dey stream tokens, so even if machine slow, e still dey feel usable because words dey appear as you dey read. Image go appear only when the last step finish. Slow means say you go dey look progress bar.

The hardware ladder, with real numbers

The block below dey hold typical figures for one SDXL image for 1024x1024, 30 steps, Euler sampler, as of July 2026. Treat dem as order of magnitude. Your sampler, step count and resolution fit change dem.

[
  {
    "label": "8 vCPU VPS, no GPU",
    "seconds_per_image": 780
  },
  {
    "label": "8GB VRAM GPU",
    "seconds_per_image": 32
  },
  {
    "label": "12GB VRAM GPU",
    "seconds_per_image": 18
  },
  {
    "label": "24GB VRAM GPU",
    "seconds_per_image": 9
  }
]

The CPU row na 780 seconds, about thirteen minutes. The 24 GB card na 9 seconds. Na this gap you dey pay to reduce.

Read the ladder like this. Below 8 GB of VRAM, SDXL still dey run, because ComfyUI dey offload layers go system RAM automatically and e fit use card wey get as little as 1 GB. Offloading dey add time for every step, so 6 GB card dey take closer to one minute per image than thirty seconds. For 8 GB, the base model fit enter and render dey comfortable. For 12 GB, you fit keep one or two ControlNet together with the checkpoint without offloading. For 24 GB, you fit run SDXL plus the refiner plus upscaling for one workflow, and you fit start to train LoRA adapters, wey need much more memory than generating.

Wetin CPU VPS fit do and wetin e no fit do

E fit do pass wetin people expect, and e fit do less than marketing dey imply. Make you clear about the limit.

CPU VPS fit install ComfyUI, serve the web interface, keep your model library, run the queue, and generate images without any GPU at all. With Stable Diffusion 1.5 for 512x512 and 20 steps, expect roughly 60 to 150 seconds for each image on 8 modern vCPUs with 16 GB of RAM. For batch job wey go run overnight, or low-volume image endpoint behind queue, this one genuinely okay.

CPU VPS no fit give you interactive work. Prompt iteration mean twenty renders for one hour, and if each one take thirteen minutes, you go get four. E still no fit train. LoRA fine-tuning for CPU dey take days, no be hours, so treat am as unavailable.

The memory rule for CPU-only different from the GPU rule. The weights load into system RAM, so you need the model size plus working space: about 16 GB of RAM for SDXL, and about 8 GB for SD 1.5. 4 GB box go start ComfyUI, then the out-of-memory killer go kill am halfway through the first render. You go see the process disappear with Killed inside dmesg and no Python traceback.

Install ComfyUI on a GPU machine

Na so upstream commands be. Start from clean Ubuntu 24.04 install wey NVIDIA driver don already dey. Confirm the driver first, because without this check, every later failure go look the same.

nvidia-smi

E suppose print table wey show your card and CUDA version. command not found, or NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver, mean say driver dey missing or kernel module no load after upgrade. Fix am before you continue, because ComfyUI go silently fall back to CPU and you go blame the software.

sudo apt update
sudo apt install -y git python3-venv python3-pip wget
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
python3 -m venv venv
. venv/bin/activate
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt

Virtual environment no be optional advice. PyTorch dey pull plenty dependencies, and to install am system-wide for machine wey dey run other things na how you go break the other thing. Verify say PyTorch fit see the card before you continue.

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

True plus your card name mean say the stack dey work. False mean say the wheel wey you install no match the driver, usually because CPU-only torch wheel don already dey cache. Reinstall with the index URL above.

Get model, and plan for disk

ComfyUI no get any weights. Put checkpoints for models/checkpoints, VAE files for models/vae, and LoRA adapters for models/loras.

cd ~/ComfyUI/models/checkpoints
wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

Always prefer .safetensors pass .ckpt. .ckpt file na pickled Python object, and when you load am, e go execute code from whoever build am. Safetensors format hold tensors only, so hostile file no fit run anything.

Storage na cost wey people dey forget. One SDXL base checkpoint na 6.94 GB. The refiner na another 6 GB. One ControlNet model dey use 1.4 to 2.5 GB, one upscaler 60 to 350 MB, and one LoRA 20 to 400 MB. Anybody wey enjoy this go download second and third base model within one week. Plan 100 GB disk for working install, and monitor outputs directory too: 1024x1024 PNG files dey 1 to 2 MB each, and unattended batch fit quietly fill small disk. Put models/ and output/ for volume wey you fit grow, and back up your workflow JSON files with something like encrypted incremental backups to object storage. You fit download the weights again. But you no fit recover the workflows wey you don tune.

Run am, then reach am safely

cd ~/ComfyUI
. venv/bin/activate
python main.py --listen 127.0.0.1 --port 8188

ComfyUI dey serve for port 8188. For box wey na CPU-only, add --cpu. Dis one dey force CPU path instead of e failing because CUDA device no dey.

Bind am to 127.0.0.1, no be 0.0.0.0. ComfyUI no get login screen and no user accounts. Anything wey reach the port fit queue jobs, read every image wey you don generate, and install custom nodes. Dis one fit execute any code for your server. Reach am through SSH tunnel from your laptop instead.

ssh -N -L 8188:127.0.0.1:8188 you@your-server

Then open http://127.0.0.1:8188 locally. If you need real multi-user access, put reverse proxy with authentication in front of am and keep the app bound to localhost. The same reasoning apply to any unauthenticated self-hosted service, and na the standard pattern for Docker Compose deployments on a VPS.

Make e dey run under systemd

Render queue wey dey die when your SSH session close no be service. Write /etc/systemd/system/comfyui.service.

[Unit]
Description=ComfyUI
After=network-online.target

[Service]
User=comfy
WorkingDirectory=/home/comfy/ComfyUI
ExecStart=/home/comfy/ComfyUI/venv/bin/python main.py --listen 127.0.0.1 --port 8188
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now comfyui
systemctl status comfyui

active (running) and log line wey dey read To see the GUI go to: http://127.0.0.1:8188 mean say e don dey run. Note say ExecStart name the interpreter inside the virtual environment directly, because systemd no dey run your shell profile and activate no dey happen.

Di pragmatic recommendation according to budget

If you wan try image generation and cost matter pass speed, use CPU VPS wey get 16 GB RAM, run SD 1.5 for 512x512, and accept say each image fit take one or two minutes. Na this one be the honest way to start, and e cost small compared with anything wey get GPU. E still be the correct place to host the model library and workflows while you dey decide.

If you dey change prompts every day, rent GPU wey get at least 12 GB VRAM by the hour from GPU cloud, and shut am down when you stop. Image generation na work wey dey happen in bursts, and idle GPU wey dem dey bill monthly na the commonest way people dey overspend for here. Keep the checkpoints for cheap block storage and mount dem.

If you dey serve other people, or you dey train LoRA adapters, you need 24 GB VRAM and machine wey you go keep. For that point, the same box usually go pay for itself if e dey run local language model too, and na this setup self-hosting LLM with Ollama describe.

Any rung wey you choose, measure your own machine before you believe any published figure. Queue the same prompt five times and read the seconds-per-iteration wey ComfyUI print for im console. That number na your real position for the ladder.

FAQ

I fit run Stable Diffusion without GPU?

Yes. ComfyUI dey run with python main.py --cpu and e dey generate real images even when graphics card no dey. Expect around 60 to 150 seconds for each image with Stable Diffusion 1.5 at 512x512, and ten to twenty minutes for SDXL at 1024x1024, on 8 modern vCPUs. E good for overnight batches and endpoints wey no get plenty traffic. E no good for prompt iteration, and training no dey practical at all.

How much VRAM I need for SDXL?

8 GB dey run SDXL well for 1024x1024. If e lower than that, ComfyUI dey offload layers go system RAM automatically and e still dey work, reach about 1 GB of VRAM, but every offloaded step dey add time. 12 GB let you keep one ControlNet together with the checkpoint, and 24 GB cover base plus refiner plus upscaling inside one workflow. Na 24 GB be the practical minimum for training LoRA adapters.

How much disk space the models need?

The SDXL base checkpoint alone na 6.94 GB, and the refiner add about 6 GB more. ControlNet models na 1.4 to 2.5 GB each, LoRA adapters na 20 to 400 MB, and upscalers fit reach 350 MB. Plan 100 GB for working install with some base models. Monitor the output directory separately, because 1024x1024 PNG files dey take 1 to 2 MB each.

E safe to expose ComfyUI on public internet?

No. ComfyUI no get any authentication, and its custom-node system dey install and run Python code from the interface. So, open port mean remote code execution for your server. Bind am to 127.0.0.1, reach am through an SSH tunnel with ssh -N -L 8188:127.0.0.1:8188 you@your-server, and put an authenticating reverse proxy in front of am if more than one person need access.

Why my render get killed without error message?

If the process disappear with Killed inside dmesg and no Python traceback show, na Linux out-of-memory killer cause am, no be ComfyUI bug. For CPU-only machine, the weights dey stay for system RAM. So SDXL need about 16 GB and SD 1.5 about 8 GB, plus working space. Add RAM, add swap, or change to smaller model and lower resolution.