Local vs Hosted Proxy for Janitor AI: Speed, Cost and Setup Compared

People read a guide on running an LLM locally and picture the whole thing: free forever, no filter, nothing leaving their computer. Every part of that is true.

It's just not the whole picture, and the gap is bigger than the guides suggest.

What those guides don't tell you is how it feels to use. So here are the numbers, most of them from people who actually run local setups.

First, a definition, because "proxy" means two different things on Janitor. Here it means a reverse API proxy: routing Janitor to a different model instead of JanitorLLM. It has nothing to do with residential IP proxies for changing your location. If that's what you came for, no amount of this will get you a model.

Local vs hosted proxy (TL;DR)

Local means running an open-source model on your own computer, with KoboldCPP or LM Studio. You then point Janitor at it through a tunnel.

Hosted means pasting an API key and a URL from a provider who runs the model for you.

The honest summary: local wins on cost and control, hosted wins on quality and speed. Most people who try local go back, and it's rarely about the money.

Is it better to use a proxy on Janitor AI?

Yes, if JanitorLLM has stopped being enough. That's the real reason anyone is reading this.

JLLM is free and needs no setup. It also queues at peak times, and its context window is short enough that it forgets the start of a long roleplay. A proxy fixes both. Which kind is the rest of this page.

The real cost of running a model locally

Nothing per message. That part is completely true, and it's the strongest argument local has.

The costs are elsewhere.

Your hardware. A 7B model needs around 8GB of RAM. A 13B needs 16GB or more. No GPU at all? CPU works, just slower. Someone on an RTX 2060 from 2019 can run an 8B model fine.

Your electricity, and your PC being on. A GPU under load pulls a few hundred watts. It's small money, but the machine has to be awake and running the model whenever you want to chat.

Your time. More on that below, because it's the real cost.

Speed: 4 to 17 tokens per second

This is the number that decides it, and almost nobody publishes it.

The author of the most detailed local LLM guide on r/JanitorAI_Official benchmarked their own machine, an i5 with an RTX 3060 and 32GB of RAM:

• Stheno 8B: 12 to 17 tokens per second
• MN-Violet-Lotus-12B: 4 tokens per second, which they call "slow but usable"

Four tokens a second is roughly three words a second. A 300-token reply takes over a minute. You watch it arrive.

A hosted model on the same reply finishes faster than you can read it.

Running DeepSeek at home costs $10,000

You can't, not really. Asked this directly, the same guide's author estimated you'd need a Mac Studio M3 Ultra with 512GB of unified RAM, around $10,000, and said they wouldn't recommend it.

DeepSeek V4 Flash through a hosted provider costs $0.14 per million input tokens and $0.28 per million output. A long roleplay session is a fraction of a cent.

That's the gap. Not a little cheaper, a different category.

Why can't I use a proxy on Janitor AI?

Because of this, usually, and it's the part local guides bury.

Janitor uses a Content Security Policy that stops your browser connecting to local addresses. Point it at localhost or 127.0.0.1 and it silently fails. Your model is running. Your URL is right. Nothing happens.

There are two ways around it.

A tunnel. KoboldCPP has a Remote Tunnel option that gives you a public Cloudflare URL. This is the easier path and what most people use now.

nginx plus ngrok plus a browser extension. The reference setup for LM Studio needs an nginx config, an ngrok account, and a browser extension that disables CSP. Its README says plainly: "JanitorAI blocks custom API endpoints via Content Security Policy (CSP)."

Both have the same catch. The tunnel URL changes every time you restart, so you re-paste it into Janitor each session. The permanent fixes are port forwarding, which the guide's own author calls "unsafe if not done properly", or buying a domain and routing it through Cloudflare.

Privacy, and the part local guides skip

Mostly yes, with one thing worth knowing.

Your chats never reach anyone's server, and there's no account, no policy and no filter. That's real, and it's the best reason to run local.

The caveat is the tunnel. To get past CSP you expose your local server to the internet through ngrok or Cloudflare. The CSP-bypass route goes further and asks you to disable a browser security policy, in the same browser where you're logged into everything else.

It's not a disaster. But "nothing leaves my machine" isn't quite the setup you end up with.

Is using a hosted proxy safe?

It depends entirely on who runs it, and Janitor says so itself in the settings panel:

"Janitor has no affiliation with any proxies. Your IP and chat log might be logged. Please ensure you trust the proxy."

Community reverse proxies are the ones to think hardest about. Someone is paying for those tokens out of their own pocket, and your messages route through their machine. Write nothing there you'd mind being read.

What to check before pasting a key anywhere:

• Is there a published policy on prompt retention?
• Is the operator identifiable?
• Does it need an account tied to your email?

The quality gap nobody quantifies

More than the parameter counts suggest, and local's own advocates say so.

A local setup runs 8B to 22B models. Stheno 8B and MN-Violet-Lotus-12B are the usual roleplay picks, and they're genuinely good for their size. Most are fine-tuned on fiction rather than assistant transcripts, so they handle a scene better than their benchmarks suggest.

But the guide that recommends them also says of its own NSFW examples: "Slop tokens are to be expected." The fix it suggests is banning tokens in a different frontend, because Janitor can't do it.

Hosted models are a generation ahead on coherence. They track who's in a room. They remember what happened forty messages ago. They don't collapse into repeated phrasing, which over a long roleplay is most of the experience.

Can you run a local model on a phone?

No. This is the question asked most often under local guides and answered least.

Local needs a PC running the model, awake, with a tunnel open. If you chat on a phone or tablet, hosted is your only option. Most people on Janitor are on mobile.

You can run the model on a desktop at home and connect from your phone through the tunnel, but the desktop has to stay on.

Picking your side

Go local if you have a desktop with a GPU, you're comfortable with config files and tunnels, you want zero per-message cost, and you'd rather have total control than the best writing. Start with KoboldCPP and Stheno 8B at Q5_K_M.

Go hosted if you're on mobile, you want to start in two minutes, or you want models a home PC can't run.

The thing most people get wrong is assuming the choice is about money. It usually isn't. Local costs nothing per message and hosted costs fractions of a cent, so after a year of heavy roleplay you're comparing zero against a few dollars.

What you're really choosing between is a 12B model at four tokens a second that you control completely, and a frontier model that answers instantly and belongs to someone else.

Where NanoGPT fits

If you land on hosted, the two things that pushed you toward local in the first place are worth keeping.

No filter. GLM 5.3 Flash Uncensored is $0.20 in and $0.80 out per million tokens, with nothing to jailbreak and no prefill tricks.

No logging. We don't log or retain prompts, and the API works without an account at all, so there's no email attached to a session.

And DeepSeek V4 Flash, the model that would cost you $10,000 of hardware to approximate at home, is $0.14 and $0.28. Ten cents of credit gets you started, which is less than the electricity for a weekend of local roleplay.