Build your own AI agent on a VPS

What an AI agent actually is

An AI agent is a loop wrapped around a language model. The model reads the situation, decides on one action, your code carries that action out, the result goes back to the model, and the loop runs again until the task is done. That is the whole idea. A plain chatbot answers once and stops. An agent keeps going, taking real actions between its own turns, until it reaches a goal you gave it.

The action is the important part. On its own a language model only produces text. It cannot read a file, call an API, or run a command. An agent gives the model a set of tools it is allowed to use, and a way to ask for them. When the model wants to search the web or write a file, it does not do the work itself. It emits a structured request, your code runs the tool, and the answer comes back as the next thing the model reads. The model supplies the judgement; your server supplies the hands.

Not every task needs an agent, and reaching for one by default is a common mistake. If the steps are known in advance, a plain script is simpler, faster, and more reliable. "Fetch this page every hour and email me the price" is a scheduled job, not an agent. Build an agent when the path is not fixed ahead of time, when the model has to look at what it finds and decide what to do next. The cost of an agent is unpredictability, so only pay it when the flexibility earns its keep.

Tools: how an agent acts

A tool is any capability you hand the model, described well enough that it knows when to reach for it. Read a file, run a shell command, query a database, send a message: each is a tool with a name, a short description, and a list of inputs. You define the tools; the model decides when to call them.

The mechanism is the same everywhere, whatever model you use. The model returns a structured request that names a tool and fills in its inputs. Your code sees that request, runs the matching function, and sends the result back in the next turn. The model reads the result and either calls another tool or writes its final answer. Function calling is the plumbing under every agent, and the loop that drives it is only a few lines of ordinary code.

This is also where your control lives. The model can ask to run a command, but nothing runs until your code chooses to run it. That gap is where you put approval prompts for dangerous actions, limits on what a tool may touch, and a log of everything the agent did. An agent is only as safe as the tools you give it and the checks you put in front of them.

MCP: a standard way to connect tools

Writing a fresh integration for every service by hand gets old fast. The Model Context Protocol, or MCP, is an open standard that solves this. Instead of coding a new tool for your files, your database, and your issue tracker, you point the agent at an MCP server that already exposes those things as tools. The agent speaks one protocol; the server does the work of talking to the real system.

The payoff is reuse. An MCP server someone else wrote for a service you use becomes available to your agent with no new integration code, and a server you write is usable by any agent that speaks the protocol. On a VPS this matters, because you can run MCP servers as their own small services next to the agent, each with only the access it needs. I cover the setup in running MCP servers on a VPS.

Memory and retrieval

A language model has no memory of its own between calls. Everything it knows about the current task has to be handed to it each turn. For a short job that is fine, because the whole conversation fits in one request. For anything longer you have to manage memory yourself, and there are two patterns worth knowing.

The first is a scratchpad. You give the agent a file it can read and write, and tell it to record what it learns as it goes. On the next turn, or the next session, it reads the file back and picks up where it left off. This is memory as a plain document, and it works because the agent treats the file as just another tool.

The second is retrieval. When the agent needs knowledge from a large body of documents that would never fit in one request, you store those documents in a searchable form and pull only the relevant pieces into the model's view when they are needed. This pattern is called retrieval-augmented generation, or RAG. The agent asks a question, your code finds the handful of matching passages, and only those go to the model. The store lives on your server, so your private documents never leave it.

Many agents, one coordinator

One agent with many tools works for most tasks. When a job is large or naturally splits into parts, a different shape helps: a coordinator agent that delegates to specialised sub-agents. The coordinator breaks the goal into pieces, hands each piece to a sub-agent built for that kind of work, and combines the results.

The gain is focus. A sub-agent with a narrow job and a small tool set makes better decisions than one generalist juggling everything, and independent pieces can run at the same time. The cost is coordination, which is real, so keep to a single agent until a task clearly wants more. Start simple, and add agents only when one is plainly straining.

Self-hosted or hosted: which model runs your agent

The model is the one part of an agent you do not have to run yourself, and choosing where it lives is the biggest decision you will make. A hosted model, reached over an API, gives you the strongest reasoning with nothing to operate: you send text, you get text back. A self-hosted model runs on your own server, which keeps every request private, costs a flat price instead of a fee per token, and never depends on anyone else staying up. The trade is capability and effort. The best hosted models are ahead of what you can run yourself, and running your own means feeding it enough memory to fit.

That last point is the practical catch. A model has to fit in your server's memory, and if you use a GPU, in its video memory. A model too large for the hardware will not load. Before you plan a self-hosted agent, check whether the model you want fits the machine you have:

If the numbers do not fit, you have three moves: pick a smaller model, use a more aggressive quantization to shrink it, or use a hosted API for the reasoning and keep only your tools and data on the server. Many self-hosted agents start with a local model through Ollama on a VPS and fall back to a hosted API for the hardest steps.

The server is the dangerous part

An agent that can run shell commands and write files is powerful, and that is exactly why it is dangerous. The model's judgement is good but not perfect, and a bad instruction, a bug, or a hostile input can turn a helpful agent into one that deletes the wrong thing or leaks a secret. The security work is not optional, and on a server it is the part that matters most.

A few habits carry most of the weight. Run the agent as a dedicated unprivileged user, never as root, so a mistake has a ceiling; the same reasoning is in running services as an unprivileged user. Keep its secrets, such as API keys, out of the code and readable only by that user. And sandbox the tools that touch the system, so the agent can only reach what it truly needs. For a worked example of hardening a real self-hosted agent, see running OpenClaw safely on a VPS. If you would rather use a hosted model for the intelligence, the companion guide on building an agent with Claude on a VPS takes the same ideas and puts a specific model behind them.

For a worked example, building an OpenClaw-style personal agent applies these pieces, and if you would rather run a finished one, start with self-hosting Hermes Agent on a VPS or running Agent Zero on your own server.

FAQ

What is the difference between an AI agent and a chatbot?

A chatbot answers a message and stops. An agent runs a loop: the model decides on an action, your code carries it out, the result goes back to the model, and it repeats until the task is finished. The difference is that an agent takes real actions between its turns, calling tools to read files, run commands, or query services, rather than only producing text.

Do I need a GPU to run an AI agent on a VPS?

Only if you self-host the model. The agent loop, the tools, and the memory are ordinary code that runs fine on a normal VPS with no GPU. A GPU matters when you want to run the language model on your own hardware, because the model has to fit in memory. If you use a hosted model over an API, the heavy computation happens elsewhere and a modest VPS is enough.

What is MCP and do I need it to build an agent?

MCP, the Model Context Protocol, is an open standard for connecting an agent to tools and data sources. You do not strictly need it, since you can write each tool by hand. MCP saves you that work by letting you reuse existing servers for common services and expose your own systems once for any agent to use. It is a convenience that becomes worth it as the number of integrations grows.

Is it safe to give an AI agent access to my server?

It can be, if you contain it. An agent that runs commands is only as safe as the account it runs under and the tools you allow. Run it as an unprivileged user, keep its secrets out of reach, sandbox the tools that touch the filesystem, and require approval for actions that are hard to undo. Treat the agent as untrusted code that happens to be clever, and give it only what the task needs.