⌨️
The council recommends
On: "Please suggest ideal free LLMs available for download from Hugging Face that are suitable for use in a Privacy AI Keyboard. I need two models: a general default model with tool-calling capabilities and a model specifically for AI instructions. I am using an iPhone 15 Pro Max, so please recommend the most capable GGUF or MLX models that will perform well on this device. Include direct links to download the recommended models, as well as the precise metadata, quantization, and specifications for each specific model version you suggest."
Top pick
Qwen2.5-3B-Instruct (Q4_K_M) as default tool-caller + Phi-3.5-mini-instruct (Q4_K_M) for instructions
All three experts back Qwen2.5-3B-Instruct at GGUF Q4_K_M as the default tool-calling model — it has the strongest, best-documented function-calling of any small open model, is Apache-2.0, ~2.1GB at Q4_K_M, and fits the A17 Pro's 8GB RAM. For the instruction/rewriting model, go with Phi-3.5-mini-instruct Q4_K_M (MIT, ~3.8B, cap context to 4k-8k on-device), which two of three experts favor for its superior instruction following. Prefer official/well-known repos: Qwen/Qwen2.5-3B-Instruct-GGUF and bartowski/Phi-3.5-mini-instruct-GGUF. Critically, treat all exact download URLs, byte-sizes, and quant file names as unverified — confirm the current file names directly on the Hugging Face model pages before downloading, since these can drift and links can be wrong. If your keyboard uses MLX Swift, grab the mlx-community 4-bit builds instead of GGUF.
Runner-up
Llama-3.2-3B-Instruct (Q4_K_M) as the instruction model instead of Phi-3.5-mini — lighter and fast, with a Llama-3.2-1B Q4_K_M fallback (~0.8GB) if you need lower memory/latency.
Wildcard
Use MLX 4-bit community builds (mlx-community) rather than GGUF if your app is built on MLX Swift — they tend to run faster and more memory-efficiently on Apple Silicon than llama.cpp GGUF for the same model.
Best time
Before committing, first validate anthropic's critical caveat: iOS keyboard extensions have very tight RAM limits (often only tens to a few hundred MB), so a 3B model will likely need to run in the host app rather than the extension itself. Prototype memory usage on-device first, verify live Hugging Face file names, then download and benchmark latency/battery.
78%
aligned
All three experts converge on Qwen2.5-3B-Instruct Q4_K_M as the default tool-caller, with the main split being the instruction model (Phi-3.5-mini vs Llama-3.2-3B) and how seriously to treat iOS keyboard-extension memory limits.
Your turnFacing a decision like this?Council puts 3 AI models on your question and gives you one verdict. Free, no signup.
Where they agree
✓Qwen2.5-3B-Instruct in GGUF Q4_K_M is the best small open model for the default tool-calling role at this size.
✓3B-4B parameter models at 4-bit (Q4_K_M) are the right capability tier for the iPhone 15 Pro Max, and 7B+ models should be avoided due to latency and memory pressure.
✓Q4_K_M quantization is the sweet spot balancing quality, speed, and memory for on-device keyboard inference.
Unlock the full council
All 3 advisors backed this verdict. See how each one argued it.
The 3 takes
GPT-5.5Lens · capability & concrete specs
Claude Opus 4.8Lens · deployment risk & feasibility
Gemini 3.1 Flash LiteLens · performance & user experience
Recommends Qwen2.5-3B-Instruct Q4_K_M as default tool-caller and Phi-3.5-mini-instruct Q4_K_M for instructions, emphasizing latency/battery balance on the A17 Pro.
90% confident
Related sessions
Take the council with you
Ask, follow up and revisit every verdict from your phone.
Synthesized from 3 independent expert passes. Treat as informed input, not a guarantee. Your own judgment is the final vote.