Skip to content
Change language: EN
My learningAdd to Chrome
Track progress 0%

Lesson 9 5 min

Pick an on-device model that fits

On-device models run entirely in your browser, free and private, and work offline once installed. Choose one your device can handle.

Open Local AI

In the sidebar, open Local AI (or Settings › Local models). Models are grouped by task (Chat, Image classification, Speech-to-text, Embeddings…), with a search box and a filter for the engine: WebLLM, TensorFlow.js or transformers.js. You pick a model; AWFlow picks the engine.

The Local AI page: storage used by on-device models split by engine (WebLLM, TF.js, Transformers), a task list on the left (Chat, Embeddings, vision and audio tasks) and model cards on the right, such as Llama 3.2 1B Instruct with its size, variants and install state.The Local AI page: storage used by on-device models split by engine (WebLLM, TF.js, Transformers), a task list on the left (Chat, Embeddings, vision and audio tasks) and model cards on the right, such as Llama 3.2 1B Instruct with its size, variants and install state.
A model card: size, device fit and install state

Once a model is installed it works offline, costs nothing per use, and no text, image or audio leaves your device.

Read the device fit

Open a chat model, for example Llama 3.2 1B Instruct. Its variants are the same model at different precisions. Each variant shows an estimated size and a device fit: Fits comfortably, Tight fit or Exceeds memory, worked out from your device’s estimated memory and hardware support. A smaller variant trades a little quality for less memory and a faster load.

Install and test

Click Install on a variant. Downloads run in the background, one at a time, so you can keep working. When it’s done, click Test to confirm it works on this device. For a chat model, click Set default: the assistant prefers it, and new Web LLM nodes are pre-filled with it.

The detail view of Llama 3.2 1B Instruct on the Local AI page, scrolled to Use it in (Web LLM, Transformers Chat, Run Local Model) and Variants: four precisions (q4f16_1 ≈ 879 MB, q4f32_1 ≈ 1.1 GB, q0f16 ≈ 2.5 GB, q0f32 ≈ 5.0 GB), each with an Install button.The detail view of Llama 3.2 1B Instruct on the Local AI page, scrolled to Use it in (Web LLM, Transformers Chat, Run Local Model) and Variants: four precisions (q4f16_1 ≈ 879 MB, q4f32_1 ≈ 1.1 GB, q0f16 ≈ 2.5 GB, q0f32 ≈ 5.0 GB), each with an Install button.
Install a variant

If the test fails, try a smaller variant: hardware support varies between machines.

Key takeaways

  1. Local AI lists models by task, with an estimated size and a device fit for each variant.
  2. Smaller variants load faster and need less memory.
  3. Set default makes a chat model the one new Web LLM nodes are pre-filled with.