Technology November 20, 2026 9 min read

Why pay per token when you can run your own?

By Yasser Bundhoo

Why pay per token when you can run your own?

We moved our high-volume AI from pay-per-token APIs to our own GPU servers. At about 430 million tokens a month, a fixed price costs far less, and our data stays under our control. Local models aren’t yet at frontier level, so we use both.

We've always invested in the best AI tools available, including OpenAI, Anthropic and other leading providers. They're great tools, and we still use them where they make sense. But AI quickly became part of our daily production: it now generates structured data, texts, summaries and classifications for our business every day, and feeds them directly into our systems, including our ERP. The more our processes ran on AI, the more tokens we used, and the costs rose month after month. So for our core, high-volume workloads, the decision was made: run AI on our own dedicated infrastructure.

Over the past months we have built, managed and maintained that infrastructure. This report covers what it really took: finding servers in a market where GPUs are scarce, what it costs compared to paying per token, how the system works, how we keep it secure, what we learned along the way, and where we are taking it next.

Cost, availability and finding the right hosting

The first challenge wasn't technical. It was simply finding a server.

AI has created enormous demand for GPU hardware. Large companies and AI labs are buying and renting GPU capacity on a massive scale, and smaller companies feel it directly. When we started looking, many hosting providers had no suitable GPU servers available at all, others had long waiting times, and the ones that were available came at extraordinarily high prices. Industry reports from 2026 describe the same picture: rising GPU rental prices, lead times of many months for new high-end GPUs, and server prices still going up.

We needed more than just hardware. Our criteria were:

  • A dedicated GPU, reserved for us alone, with full access to the card instead of a shared slice of it.

  • A fixed monthly price, so costs stay predictable regardless of how much we use.

  • A reliable data centre with a guaranteed availability of 99.9 % and professional support.

  • Enough memory and storage to run large language models, embedding models and our own data side by side.

In the end we found dedicated servers that met all of these points: professional-grade, high-performance NVIDIA GPUs on a fixed monthly contract.

The cost comparison: tokens vs. a fixed price

This is where the decision becomes clear. Our platform currently processes around 100 million tokens per week, which is about 430 million tokens per month.

Here is what the same volume would cost with commercial AI APIs at their standard published prices (November 2026):

Option

Price per 1M tokens (input / output)

Est. per month

Est. per year

Claude Sonnet 5.5 (Anthropic)

$2 / $10

~$1,540

~$18,500

GPT-6.2 Sol (OpenAI)

$2 / $10

~$1,540

~$18,500

Claude Opus 5.5 (Anthropic)

$4 / $20

~$3,090

~$37,000

GPT-6.2 Astra (OpenAI)

$8 / $40

~$6,170

~$74,100

How it's calculated: about 430 million tokens per month, assuming 80 % input and 20 % output, which matches our workload. If the share of output were higher, the API costs would be higher too. Prices are standard list prices without discounts or taxes.

What the numbers show:

  • Our server runs on a fixed monthly price that is well below even the mid-range options in this table, and a small fraction of what the top models would cost.

  • With an API, every additional request costs more. On our server, more usage costs nothing extra until the hardware is fully used, and it isn't.

  • The cost is predictable. No surprise invoices when usage spikes, and no exposure to price changes by a provider.

To be fair: the very cheapest small cloud models cost less per token. But they are much smaller models, they would need testing for our tasks, and every request would still send our data to a third party.

What we built

  • Two dedicated GPU servers, set up and secured from scratch: operating system, access, firewall and monitoring.

  • Open-weight AI models running directly on the GPU.

  • Our own AI platform, "Merkur": a dashboard where we start and stop models, manage access keys, track usage and costs, manage versioned prompts and review AI results.

  • One standard interface for all applications, so switching an existing tool from an external provider to our server is mostly a configuration change.

Privacy and security

Privacy and security were built in from day one:

  • Data protection: for these workloads, candidate and client data is processed on servers reserved for us and is never shared with AI providers. The setup was designed with GDPR and Swiss data protection in mind.

  • Hardened access: key-only logins, strict firewalls and intrusion protection.

  • Vulnerability scanning and penetration testing to identify weaknesses proactively. A scan runs automatically every night and the results appear directly in our dashboard.

  • Monitoring and instant alerts to help us detect problems early and respond before users are affected.

Patching in a time of constant attacks

Running our own servers means nobody patches them for us, and the threat is real. 2026 has shown that even the most widely used and trusted software is not safe:

  • NGINX (May 2026): a critical flaw nicknamed "NGINX Rift", hidden in the code for around 16 years, was attacked within days of being published. NGINX is one of the most widely used web servers in the world.

  • PHP (October 2026): two critical flaws in PHP itself, known as "PHP2Shell", were actively exploited worldwide shortly after they became public. They allowed attackers to take over unpatched web servers remotely, putting millions of websites at risk.

  • Java (November 2026): a critical flaw in the Java runtime, "Java4Shell", was disclosed together with Oracle's November security update. It could allow attackers to run their own code in Java applications.

  • The Linux kernel, container platforms and system libraries receive security fixes every month as well.

The pattern is always the same: once a flaw is public, attackers move within days. The only real protection is to patch quickly and consistently. Our approach:

  • Security updates are applied regularly, and critical fixes as soon as they are released.

  • Every layer is covered: the operating system and kernel, the web server, the programming languages and frameworks, the databases and all the containers the platform runs in.

  • Sensitive updates are planned. Updates to the GPU driver, the system kernel and the container platform are tested and installed in planned maintenance windows, because one faulty driver update can take the GPU offline.

  • Nightly scans show what is still open, so nothing is forgotten and every critical finding gets attention.

Our alert bot

We built our own bot that watches the platform around the clock. As soon as a request through our AI gateway fails, for example because a model is not responding or an application sends invalid requests, the bot sends an alert to our team chat with the details. The same alerts appear in a notification bell inside our dashboard. Problems reach us within seconds, wherever we are, instead of being discovered hours later by a user.

Local is not the same as frontier

It is important to be honest about one thing: running AI locally does not mean we are on par with the frontier models.

The most advanced closed-source models from providers like OpenAI and Anthropic are trained with enormous resources and still lead on the hardest tasks, such as complex reasoning, difficult code and long, multi-step work. The open-source models we run are smaller and, for those tasks, they are not equal.

But for our core workloads, open models deliver the quality we need. That is why we use both: frontier cloud models where they add the most value, and our own infrastructure for high-volume everyday work.

The gap is also moving in our favour. Every few months new and better open models are trained and released, often reaching the level that only closed models had a year or two earlier. Every time that happens, our local workflow gets better: we can test a new model on our own tasks and switch to it without changing our infrastructure, our applications or our costs.

Future development

System design and scaling

The platform was designed from the start so that it can grow. The next steps are to spread the workload across both servers, to run more requests in parallel on each GPU, and to use smaller, specialised models for simple tasks so that the large model stays free for the demanding ones.

Data recovery and data management

As more business data runs through the platform, protecting it becomes as important as processing it. Our priorities are:

  • Automatic, encrypted backups stored outside the server, in a separate location within Europe.

  • Regular restore tests, because a backup only counts once it has been successfully restored.

  • Clear rules for the data lifecycle: what is kept, for how long, who can access it, and when it is deleted, in line with GDPR.

  • A documented recovery plan, so that after a hardware failure the platform can be brought back quickly and in a defined way.

Zero-downtime deployment

Today, some updates still require a short restart, which is why we plan them in maintenance windows. The goal is zero-downtime deployment: the new version starts alongside the old one, is checked automatically, and only takes over the traffic once it is healthy. If anything goes wrong, traffic stays on the old version. This lets us ship improvements and security fixes at any time without our users or the ERP noticing.

Looking further ahead: a local GPU cluster

Today we rent dedicated servers. In the long run, we are thinking about our own GPU cluster: several GPUs working together as one system. A cluster would let us run larger and more capable models, serve many more requests at the same time, and keep running even when one machine is being maintained or fails. Combined with better open models every year, this would bring us even closer to frontier quality, with our data still fully under our own control.

Whether and when this makes sense depends on how our usage grows and how the GPU market develops. But the foundation we have built, our own platform and the experience of running AI in production, makes that next step possible.

What we've learned

  • GPU memory is a budget. Every model has to earn its place.

  • Small settings, big effects. One parameter can decide whether results are reliable or random.

  • Prompts are code. Version them, test them, review them.

  • Measure before you optimise. Test whether a better prompt is enough before investing in fine-tuning.

  • A live system changes everything. Once other systems depend on your server, every restart needs a plan.

  • Security never stops. Patching, scanning and monitoring are daily work, not a one-time setup.

The result

The best of both worlds: leading cloud AI where it adds the most value, and our own dedicated infrastructure for everyday work. For our high-volume workloads that means lower and predictable costs, full control over our data and no dependence on a single AI provider, on a platform designed to grow with us.

Running AI this way isn't plug and play. Finding the hardware was hard, and running it takes real expertise and discipline every day. But we would make the same decision again.

How is your company handling AI: cloud, your own infrastructure, or a mix of both? We'd love to hear how others are approaching it.


Recent blog posts

Tips, trends and insights about jobs, careers and the market.

View All
Technology November 20, 2026

Why pay per token when you can run your own?

We moved our high-volume AI from pay-per-token APIs to our own GPU servers. At about 430 million tokens a month, a fixed price costs far less, and our data stays under our control. Local models aren’t yet at frontier level, so we use both.

Read article
Technology November 10, 2026

Not your keys, not your coins: Locke, Epictetus and the $6.3 billion question

This post analyzes the $6.3B Binance hack via John Locke (property via labor) and Epictetus (Stoic control). It argues that leaving crypto on an exchange means holding a promise, not the asset. True ownership requires self-custody of private keys, blending Locke's property rights with Epictetus's advice to prepare for the worst and accept only what is truly in your control.

Read article
Technology October 26, 2026

The Cartesian Demon in Our Servers: What PHP’s Black October Reveals About Cybersecurity

The "PHP2Shell" vulnerabilities proved that a green security dashboard is an illusion if the underlying PHP interpreter is compromised. Drawing on Descartes’ philosophy of radical doubt, this post explores why we must apply "Cartesian doubt" to the foundational layers of our tech stack. Stop blindly trusting your infrastructure—check your PHP version, audit your setup, and question your foundations before the next exploit does.

Read article
Technology October 2, 2026

What actually is Artificial Intelligence

Artificial Intelligence is much older than ChatGPT. This article explores its fascinating history, from ancient mechanical inventions to modern AI, while explaining the differences between AI, Machine Learning, Deep Learning and Generative AI. It also highlights the importance of understanding AI's limitations, biases and potential inaccuracies.

Read article
General September 17, 2026

Judgement vs. Efficiency

Automation makes processes faster, reduces costs and improves efficiency, but relying on it too heavily can weaken human judgment. This article explores the importance of balancing technological efficiency with human empathy, accountability and oversight, highlighting why people must remain in control of decisions that truly matter.

Read article