On-device AI vs cloud AI, in plain terms

"Runs on your device" has become a marketing phrase, which means it is worth understanding what it actually describes. The difference between on-device and cloud AI is not a matter of degree or of how carefully a company handles your data. It is a difference in where the computation physically happens, and everything else follows from that.

What the two actually mean

An AI model is a large file of numbers plus code that runs it. To use one, the model and your data have to be in the same place. There are only two ways to arrange that.

Cloud AIkeeps the model on a company's servers. Your data travels to it. You send a file or some text, their hardware does the work, and the result comes back. This is how most AI products work.

On-device AI sends the model to you instead. The model file downloads to your computer or phone once, and then runs on your own processor or graphics card. Your data never goes anywhere, because there is nowhere for it to go.

Everything people care about, privacy, speed, cost, offline capability, follows mechanically from that one architectural choice.

Why cloud became the default

Not because companies wanted your data, at least not originally. Because for years it was the only thing that worked.

Capable models were enormous, needing more memory than a laptop had and specialized hardware most people did not own. Running one locally was simply not possible. Centralizing the model on powerful servers was the only way to make it available to anyone without a workstation.

Once that architecture existed, the incentives followed. If data flows through your servers anyway, retaining it is nearly free, and it becomes useful for improving models and measuring usage. The privacy tradeoff was mostly a consequence of the engineering, not the goal of it.

What changed

Three things shifted at roughly the same time.

• Models got smaller without getting much worse. Techniques for compressing models, particularly quantization, which stores the numbers at lower precision, cut memory requirements dramatically for a modest accuracy cost.
• Ordinary hardware got much better at this specific kind of maths. The graphics hardware in a normal laptop or phone is genuinely good at the operations models need.
• Browsers gained access to that hardware. WebAssembly let heavy computation run at near-native speed in a web page, and WebGPU opened up the graphics card. A web page can now do work that used to require installing software.

The result is that a useful speech recognition model is now tens of megabytes and runs in a browser tab. That was not true a few years ago.

The honest tradeoffs

On-device is not simply better. It wins decisively on some axes and loses on others, and anyone telling you otherwise is selling something.

Where on-device wins:

• Privacy, categorically.Not "we delete it after 30 days" but "it was never transmitted." There is no retention policy to trust, no breach that could expose it, and no vendor that could be compelled to hand it over.
• Works offline. Once the model is cached, no connection is needed.
• No per-use cost. Your hardware is already paid for, so there is no meter running and no usage limit to enforce.
• No queue.Nobody else's demand affects you.
• Verifiable. You can watch your own network traffic and confirm the claim. A server-side promise can only be audited, never observed.

Where cloud wins:

• Raw capability. The largest models will not fit on your device, and on genuinely difficult input they are better.
• Consistency. A server performs the same for everyone. On-device performance depends on the machine, which means a great experience on a gaming laptop and a slow one on an old phone.
• No first-run download. Cloud tools start instantly. On-device tools make you wait once.
• Scale. A three hour recording is nothing to a server and a real problem for a browser tab with a memory ceiling.
• Improves without you doing anything. Server-side models get upgraded silently.

How to tell which one a tool is using

Marketing copy will not tell you reliably, because "secure", "private", and "encrypted" are all compatible with uploading everything. Two tests settle it:

1. Watch the network.Open your browser's developer tools, select the Network tab, and use the tool. If your file is being uploaded, you will see a large outbound request. Look at what is going out, not what is coming in: a large download is the model arriving, which is the opposite of a problem.
2. Disconnect. Use the tool once so anything cacheable is cached, then turn off your internet and use it again. Nothing server-based can survive that. This test cannot be faked.

A useful question to ask any vendor: how long is my data retained, and is it used for training? The answers are revealing, and so is a vendor who cannot answer specifically.

Which to choose

The deciding question is not which is more advanced. It is whether you could defend having uploaded this particular file.

For a conference talk, a podcast, or a lecture you plan to publish, upload it. The content is not confidential, the cloud service will be faster and handle a three hour file without complaint, and its accuracy on messy audio will likely be better.

For a client interview, a therapy session, an HR complaint, a source conversation, or a board discussion that has not been announced, the calculation inverts entirely. There, the constraint is not accuracy or convenience, and the realistic alternative to an on-device tool is usually not a better tool. It is typing it out by hand.

That is the gap this site exists to fill. If you want the mechanics of how a browser does this, how it works walks through the entire pipeline, and private transcription covers the questions worth asking any transcription vendor.