Two years ago, running a language model in a browser was a science project. Today, it is a production reality. WebGPU — the browser API that gives JavaScript direct access to your GPU — combined with WebAssembly has turned the browser tab into a capable AI runtime. No install. No server. No cloud.
What changed
Three things converged:
- WebGPU shipped in Chrome, Edge, and Safari 18. Firefox support is rolling out. The API gives web pages direct GPU compute access — the same hardware a native app uses for ML inference.
- Small models got good enough. Qwen 2.5 (1.5B–7B), Llama 3.2 (1B–3B), and Phi-3 run at usable speeds on consumer GPUs. They are not GPT-4, but they do not need to be for summarization, tagging, and transcription.
- WebLLM and transformers.js matured. These libraries load model weights as ArrayBuffer chunks, compile shaders for your specific GPU, and cache everything in IndexedDB. First load takes time. Second load is instant.
What runs in the browser today
LLM chat
Models like Qwen 2.5 (1.5B) generate text at 15–40 tokens per second on a mid-range GPU. That is faster than reading speed. Glassy uses this for in-note AI — summarize, expand, rewrite, tag — without sending your draft to a server.
Speech transcription
Whisper (the OpenAI model, running locally via Whisper.cpp compiled to WASM) transcribes audio in real-time on most machines. Glassy's Voice Studio uses this for lecture capture, meeting notes, and voice journaling. The audio never leaves your device.
Semantic embeddings
Models like mxbai/MiniLM and all-MiniLM-L6-v2 generate vector embeddings entirely in-browser. These power semantic search — "find notes about the same topic even if they use different words." The embedding model is about 90 MB on first load, then cached and fully offline.
Entity extraction and GraphRAG
Projects like Nebula demonstrate that you can build entity knowledge graphs in the browser — extracting people, projects, and relationships from notes, then using GraphRAG to answer questions by walking those relationships. All in a single tab.
The privacy argument
Most AI note apps ship your content to a cloud vector database and a hosted model, then ask you to trust their privacy policy. The browser-native approach is structurally different: there is nowhere for your data to go. Notes, embeddings, and the search index live in local browser storage. No sync service, no account system, no server to breach.
For sensitive content — therapy notes, medical records, client data, anything you would not paste into ChatGPT — this is the whole point. Structural privacy is not a policy. It is an architecture.
The limits
Browser AI is not a replacement for cloud models. The constraints are real:
- Model size — You are limited to models under ~4 GB. GPT-4 class reasoning is not happening in your tab.
- First-load download — The initial model download is 500 MB to 2 GB. After that, it is cached. But the first time takes patience.
- GPU requirements — You need a discrete GPU or Apple Silicon. Older integrated graphics struggle.
- Memory pressure — Running a 2 GB model alongside 50 browser tabs will crash. Tabs get evicted.
- Firefox is still catching up — WebGPU is behind a flag in Firefox as of mid-2026.
How Glassy uses WebGPU
Glassy runs three models locally in your browser:
- Qwen 2.5 (1.5B) — In-note AI: summarize, expand, rewrite, auto-tag. Loads on first use, cached after.
- Whisper (base.en) — Voice transcription for Voice Studio. Real-time on most hardware.
- mxbai/MiniLM — Semantic embeddings for hybrid search and Related Notes. 384-dimensional vectors, ~90 MB, cached after first load.
Cloud AI (GPT-4, Claude) is available as an opt-in for harder work — long-form generation, complex reasoning, multi-document synthesis. But the default is local. Your drafts, your audio, your notes stay on your machine unless you explicitly choose otherwise. For a deeper look at where the line between local and cloud is drawn, see Local AI vs Cloud AI: what stays on your machine.
The browser is the runtime
The shift from "install an app" to "open a tab" is not just about convenience. It is about trust. When AI runs in your browser, you can verify what it does. You can open DevTools and watch the network tab. No traffic means no data leaving. That is a stronger guarantee than any privacy policy.
Glassy was built on this premise from day one. The browser is not a thin client for a cloud AI service. It is the runtime.