Best Small Models for Text Classification on a Laptop (2026)
By @pickwiseSmall models for text classification you can run on a laptop: spam, intent, routing and moderation with no API bill, picked from independent tests by Red Hat's AI safety team, Oliver Borchers and the BTZSC benchmark.
EmbeddingGemma 308M, plus a logistic regression
Turn each text into an embedding and train a logistic regression on a few dozen labeled examples per class: in Borchers' test that recipe, with a larger embedder, matched Jev. EmbeddingGemma is the small embedder to use, scoring 87.6 on MTEB's English classification tasks, about nine points above bge-large. The catch is Google's Gemma license, which gates the download. Pricing: free, under Google's Gemma terms.
- Type
- Embedding model + simple classifier
- Labels needed
- A few dozen per class (shown with a bigger embedder)
- Runs on
- Phones, laptops and desktops, per Google
- Tested on
- MacBook Pro M1 Max: 33.5 embeddings a second (Q8)
- Memory
- Under 200 MB RAM when quantized
- Size
- 308M parameters
- License
- Gemma terms (gated)
- Context
- 2,048 tokens
DeBERTa-v3, fine-tuned (base or large)
The model behind the Made with Jev build, and fine-tuned it still edges out newer encoders on accuracy. In Red Hat's prompt-injection test a fine-tuned 200M DeBERTa scored 89.0% in 54 ms on an M1, ahead of Jev's 86.4%. It reads only 512 tokens, works in English only, and needs a few hundred labeled examples before it's useful. Pricing: free (MIT).
ModernBERT, fine-tuned (base or large)
Reads up to 8,192 tokens, so it handles long support tickets, emails, logs and agent traces that would overflow DeBERTa. Fine-tuned in Borchers' test, the large version sorted fresh arXiv papers at 87.0%, a hair above Jev. Its famous speed depends on an Nvidia GPU with Flash Attention, and on a laptop it runs about as fast as DeBERTa-v3. Pricing: free (Apache 2.0).
Cloudflare Clef-flash 9B
Needs no labels, speaks Jev's API, and came closest to Jev of the open models that fit a 16 GB laptop. On a base M4 MacBook, Marco Mornati found it matched Jev on routing (90% vs 89%) and beat it on risk flags. Each decision took about 3.9 seconds there, so anything interactive wants a Pro or Max chip. Pricing: free (Apache 2.0).
- Type
- Decision model, same API as Jev
- Labels needed
- None; new categories need no retraining
- Runs on
- Apple silicon Mac, 16 GB+ (4-bit MLX)
- Tested on
- 16 GB base M4 MacBook: 3.9 s a decision
- Memory
- 6.2 GB download, 7 to 8.6 GB peak (4-bit MLX)
- Size
- 9B parameters
- License
- Apache 2.0
- Context
- 64K tokens
Qwen3-4B-Instruct-2507
If you'd rather fine-tune a small LLM than an encoder, start here. In distil labs' study of 15 small models, a LoRA-tuned Qwen3-4B-Instruct-2507 matched or beat its 120B-plus teacher on two of three classification tasks, and Apache 2.0 lets you ship it. Untuned, results swing with the prompt: one test got 53% or 78% from the newer Qwen3.5 4B depending on how the answer was read. Pricing: free (Apache 2.0).
SetFit
Hugging Face's few-shot trainer builds a classifier from 8 to 64 examples per class by fine-tuning a small sentence-transformer, and it trains on a laptop CPU in minutes. On a customer-review dataset in its paper, 8 examples per class rivaled RoBERTa-large trained on all 3,000. A plain embedding plus logistic regression scored about the same in the Model2Vec team's benchmark, so try that first. Pricing: free (Apache 2.0).
- Type
- Few-shot fine-tuning of a small embedder
- Labels needed
- As few as 8 per class
- Runs on
- CPU: trains in a few minutes, per Hugging Face
- Tested on
- Unnamed CPU: 716 texts a second (MiniLM-L6 body)
- Memory
- 70 MB (MiniLM body) or 420 MB (MPNet body)
- Size
- 110M (MPNet body) or 355M (RoBERTa-large)
- License
- Apache 2.0
- Context
- Depends on the body model
GLiClass v3
Zero-shot labels on a plain CPU: GLiClass scores every label in one pass, so a long label list barely slows it down, and the 32.7M edge model answers in about 9 ms on an M4 Max. Use it when labels change too often to train anything. Accuracy is the price: on Borchers' fresh papers the large version scored 64.3% to Jev's 86.3%. Pricing: free (Apache 2.0).
- Type
- Zero-shot classifier
- Labels needed
- None; you pass labels at run time
- Runs on
- M4 Max CPU, or a Galaxy S26 phone (edge model)
- Tested on
- 16 GB M1 Pro: 0.46 s an abstract (large)
- Memory
- 131 MB (edge) to 1.75 GB (large) download
- Size
- 32.7M (edge) to 439M (large)
- License
- Apache 2.0
- Context
- 1,024 tokens in the default pipeline
Apple Foundation Models
Built into macOS on Apple silicon Macs with Apple Intelligence, so there's no model to download and no per-token cost. Constrain the answer to an enum and it stays on your label list: in one developer's test it kept to the list in 15 of 15 runs, where free text invented categories. The context window is just 4,096 tokens, and the model changes with each OS update. Pricing: free, built into macOS.
- Type
- Built into macOS: Apple's on-device LLM
- Labels needed
- None; give it your labels as an enum
- Runs on
- Mac with M1 or later (Apple Intelligence)
- Tested on
- Desktop M4 Max Mac Studio: 265 ms to first token
- Memory
- Apple Intelligence: up to 8 to 14 GB of storage
- Size
- About 3B parameters
- License
- Apple licence, accepted once; no API key
- Context
- 4,096 tokens per session
Ettin encoders (17M to 68M)
The smallest modern encoders, from Johns Hopkins, for classifiers that must run on a CPU or in a browser tab. A community fine-tune of the 17M model scored 94.2 F1 on PII extraction against GPT-4o-mini's 78.7, and credential-line-32m below is built on the 32M. The tiny sizes give up accuracy on general tasks: the 17M averages 79.2 on GLUE. Pricing: free (MIT).
- Type
- Encoder you fine-tune on your labels
- Labels needed
- Many: credential-line-32m used 87,707
- Runs on
- x86 CPU with no GPU (CPU-only fine-tune)
- Tested on
- Desktop i7-13700K: 267/31 pairs/s (17M/68M rerankers)
- Memory
- 17M: 67 MB ONNX file, 17 MB in int8
- Size
- 17M, 32M or 68M parameters
- License
- MIT
- Context
- Up to 8K tokens
OpenAI Privacy Filter
Finds names, contact details, account numbers, private dates and secrets so you can redact text before it reaches a cloud model or a training set. It scored 0.96 F1 on the PII-Masking-300k benchmark and handled about one document a second on a 16 GB M2 MacBook Pro. One independent test caught it missing AWS keys and database connection strings, so keep a regex scanner in front. Pricing: free (Apache 2.0).
- Type
- PII detector (token classifier)
- Labels needed
- None (8 fixed labels)
- Runs on
- A laptop or a web browser, per OpenAI
- Tested on
- 16 GB M2 MacBook Pro: about 1 s a document
- Memory
- 2.8 GB weights, under 6 GB RAM
- Size
- 1.5B total, 50M active
- License
- Apache 2.0
- Context
- 128,000 tokens
Llama Prompt Guard 2 (86M)
Meta's small classifier for jailbreaks and "ignore previous instructions" attacks hidden in prompts, documents and tool output, run before an agent acts. Out of the box it's close to useless: against 629 realistic agent attacks, the default threshold caught 6. Lowered to 0.003, the 86M caught 97 to 100% at about 5% false alarms, so tune it on your own traffic. Pricing: free under the Llama 4 Community License.
credential-line-32m
Reads one line of code, config or logs and scores whether it holds a literal credential, in about 4 ms on a desktop CPU. On its author's holdout of human-written lines it scored 0.86 F1, where detect-secrets managed 0.51. It's a one-author project that misses about one credential in five, so run it beside gitleaks rather than instead of it. Pricing: free (MIT).
- Type
- Secret scanner, one line at a time
- Labels needed
- None (fixed: credential or not)
- Runs on
- CPU; card says laptop, measured on a desktop
- Tested on
- Intel desktop CPU, 4 threads: 4.1 ms a line
- Memory
- 128 MB ONNX file (fp32)
- Size
- 32M parameters
- License
- MIT
- Context
- One line, up to 128 tokens
How to choose
It started with a build on Made with Jev: a developer swapped Jev, TypeSafe's hosted decision model, for a DeBERTa large classifier on a base M1 Mac to decide, inside a Claude Code hook, whether a message was just a question, and found it just as fast and just as good for that job. Jev is the yardstick here. In Borchers' field test on 400 fresh arXiv papers it scored 86.3% with no training, for about $0.04 per million input tokens.
How to choose comes down to labels. With none, try Clef-flash on a 16 GB Mac with a Pro or Max chip, or GLiClass on anything else. With a few dozen examples per label, embed the text and train a logistic regression on top. With a few hundred, fine-tune DeBERTa-v3 or ModernBERT. No labels yet? Have Jev or a large LLM label a few hundred of your own examples, then train the small model on those.
Skip facebook/bart-large-mnli, still the default in Hugging Face's zero-shot pipeline: it dates from 2019, runs a full pass for every label, and trails newer models in both the 28-task zero-shot benchmark and BTZSC. The list is ordered by how well each model did in those tests for the work it takes to set up, with the specialists for PII, prompt injection and secrets after the general-purpose models.
Numbers checked October 5, 2026.
More guides like this
Best Local LLMs for Coding (2026)
The best local LLM for coding depends on how much memory your machine has. This list ranks open-weight models you can run yourself, by independent benchmark runs and by developers who use them in coding agents. With 24 to 32 GB, start with Qwen3.8-27B. Its 17 GB 4-bit file matched the full model on Terminal-Bench 2.1 in Quesma's test. With 128 GB of unified memory, Qwen3.8-Flash-Next posted the top score in WonderRico's benchmark. With 16 GB, gpt-oss-20b fits whole on the card, and one developer reports useful work from a 2-bit Qwen3.8-27B. With 8 GB, try Qwen3.5-9B and keep the jobs small. At 64 GB, run the 27B models at 8-bit, a 29 GB file, with a longer context. The step above that needs about 100 GB. A coding agent only works if the model emits tool calls your server can parse. Gemma 4 shipped with broken tool calling that was fixed later, and Qwen chat templates have needed community fixes, so update Ollama, LM Studio or llama.cpp before blaming the model. Agents also need room: LM Studio loads models with an 8,192-token context unless you raise it. Scores on a model card are the maker's own. A mixture-of-experts model computes with a few billion parameters per token but still has to hold all of them in memory, and context takes memory on top of the file. Qwen3.8 thinks at its highest effort by default, which is slow on home hardware. Qwen3.8-Flash-Next is not Apache licensed. Older guides still recommend Qwen2.5-Coder, Qwen3-Coder and Devstral, which general-purpose models beat in the hands-on tests cited here. The list is ordered by how many independent testers and developers recommend each model for coding, then by downloads. Numbers checked October 11, 2026.
OpenRouter Alternatives (2026): Cheaper or Self-Hosted
The best OpenRouter alternatives, hosted routers and gateways you run yourself, picked from developers who switched and reviewers who measured them. Each one still gives you one API for many AI models. Match the pick to your reason for leaving. If it is the 5.5% fee on credits, Vercel AI Gateway charges no markup, and Cloudflare AI Gateway is free when you bring your own provider keys. If you want no middleman at all, self-host LiteLLM, or Bifrost if LiteLLM's latency bothers you. If your data must stay in Europe, look at Cortecs or Requesty. If you are tired of one model behaving differently from host to host, buy from a single host such as Fireworks AI, Together AI or Groq, or use OpenCode Zen's short tested list. This market changed fast in 2026, so older guides are wrong in places. Stripe announced in August that it is buying OpenRouter. Palo Alto Networks bought Portkey in May. Helicone has been in maintenance mode since Mintlify bought it in March, and TensorZero's repository is archived, which is why neither is here. LiteLLM had two poisoned releases on PyPI in March (1.82.7 and 1.82.8), so pin your version. Self-hosting takes the gateway company out of the path, but your prompts still go to whichever model provider you call. And OpenRouter's fee is charged when you buy credits, with an $0.80 minimum, so small top-ups cost more than 5.5%. The list is ordered by how many independent developers and reviewers recommend each option, then by adoption, then by price. Prices checked October 11, 2026.