The council recommends
On: "Please suggest ideal free LLMs available for download from Hugging Face that are suitable for use in a Privacy AI Keyboard. I need two models: a general default model with tool-calling capabilities and a model specifically for AI instructions. I am using an iPhone 15 Pro Max, so please recommend the most capable GGUF or MLX models that will perform well on this device. Include direct links to download the recommended models, as well as the precise metadata, quantization, and specifications for each specific model version you suggest."
Top pick
Qwen2.5-3B-Instruct (Q4_K_M) as default tool-caller + Phi-3.5-mini-instruct (Q4_K_M) for instructions
All three experts back Qwen2.5-3B-Instruct at GGUF Q4_K_M as the default tool-calling model — it has the strongest, best-documented function-calling of any small open model, is Apache-2.0, ~2.1GB at Q4_K_M, and fits the A17 Pro's 8GB RAM. For the instruction/rewriting model, go with Phi-3.5-mini-instruct Q4_K_M (MIT, ~3.8B, cap context to 4k-8k on-device), which two of three experts favor for its superior instruction following. Prefer official/well-known repos: Qwen/Qwen2.5-3B-Instruct-GGUF and bartowski/Phi-3.5-mini-instruct-GGUF. Critically, treat all exact download URLs, byte-sizes, and quant file names as unverified — confirm the current file names directly on the Hugging Face model pages before downloading, since these can drift and links can be wrong. If your keyboard uses MLX Swift, grab the mlx-community 4-bit builds instead of GGUF.
Runner-up
Llama-3.2-3B-Instruct (Q4_K_M) as the instruction model instead of Phi-3.5-mini — lighter and fast, with a Llama-3.2-1B Q4_K_M fallback (~0.8GB) if you need lower memory/latency.
Wildcard
Use MLX 4-bit community builds (mlx-community) rather than GGUF if your app is built on MLX Swift — they tend to run faster and more memory-efficiently on Apple Silicon than llama.cpp GGUF for the same model.
Best time
Before committing, first validate anthropic's critical caveat: iOS keyboard extensions have very tight RAM limits (often only tens to a few hundred MB), so a 3B model will likely need to run in the host app rather than the extension itself. Prototype memory usage on-device first, verify live Hugging Face file names, then download and benchmark latency/battery.
78%
aligned
All three experts converge on Qwen2.5-3B-Instruct Q4_K_M as the default tool-caller, with the main split being the instruction model (Phi-3.5-mini vs Llama-3.2-3B) and how seriously to treat iOS keyboard-extension memory limits.
Where they agree
✓Qwen2.5-3B-Instruct in GGUF Q4_K_M is the best small open model for the default tool-calling role at this size.
✓3B-4B parameter models at 4-bit (Q4_K_M) are the right capability tier for the iPhone 15 Pro Max, and 7B+ models should be avoided due to latency and memory pressure.
✓Q4_K_M quantization is the sweet spot balancing quality, speed, and memory for on-device keyboard inference.
Where they diverge
↔Instruction model: openai and google recommend Phi-3.5-mini-instruct, while anthropic prefers Llama-3.2-3B-Instruct (or a lighter 1B build).
↔Anthropic strongly flags that iOS keyboard extensions have tight RAM limits (tens to a few hundred MB), meaning a 3B model may only run in the host app, not the extension itself — a caveat the others do not raise.
↔Repository source: openai/google cite official Qwen and microsoft GGUF repos, while anthropic points to bartowski and mlx-community and warns against trusting exact URLs/byte-sizes without live verification.
↔Confidence varies widely: google 95 and openai 86 are highly confident in exact links, whereas anthropic is at 62 and explicitly declines to supply precise URLs to avoid hallucination.
The 3 takes
GPT-5.5Lens · capability & concrete specs
Recommends Qwen2.5-3B-Instruct Q4_K_M as the default tool-caller and Phi-3.5-mini-instruct Q4_K_M for instructions, with exact repos, params, and context caps.
84% confident
Claude Opus 4.8Lens · deployment risk & feasibility
Agrees Qwen2.5-3B is the best tool-caller and suggests Llama-3.2-3B (or lighter 1B) for instructions, but warns that iOS keyboard-extension RAM limits may block a 3B model from running in the extension itself.
64% confident
Gemini 3.1 Flash LiteLens · performance & user experience
Recommends Qwen2.5-3B-Instruct Q4_K_M as default tool-caller and Phi-3.5-mini-instruct Q4_K_M for instructions, emphasizing latency/battery balance on the A17 Pro.
90% confident
Synthesized from 3 independent expert passes. Treat as informed input, not a guarantee — your own judgment is the final vote.