Skip to content

feat(ai-providers): add llama.cpp and MLX as local providers#1938

Merged
datlechin merged 1 commit into
mainfrom
feat/local-ai-providers-llamacpp-mlx
Jul 21, 2026
Merged

feat(ai-providers): add llama.cpp and MLX as local providers#1938
datlechin merged 1 commit into
mainfrom
feat/local-ai-providers-llamacpp-mlx

Conversation

@datlechin

Copy link
Copy Markdown
Member

Closes #1777.

What

Adds llama.cpp and Apple MLX as local AI providers, alongside the existing Ollama and custom OpenAI-compatible options.

Both llama-server (llama.cpp) and mlx_lm.server (Apple's mlx-lm) expose OpenAI-compatible HTTP servers (/v1/chat/completions, /v1/models, SSE streaming, no auth by default). TablePro's OpenAICompatibleProvider already drives exactly that contract, so this is an additive registry change, not new transport code. Ollama is actually the special-cased provider (native /api/chat, /api/tags, newline-JSON streaming); llama.cpp and MLX are more purely OpenAI-compatible and ride the standard path.

Changes

  • AIModels.swift: new AIProviderType cases .llamaCpp and .mlx (display names llama.cpp / MLX, default endpoint http://localhost:8080, no API key, distinct SF Symbols).
  • AIProviderRegistration.swift: register both in the OpenAI-compatible loop, so they appear in the Add Provider menu with endpoint config and model fetching.
  • AISettingsView.swift: generalize the "Local" status label from Ollama-only to any local provider with no auth.
  • Docs and CHANGELOG updated. New LocalProviderRegistrationTests mirroring the xAI registration test.

Design notes

  • The generic custom provider remains the escape hatch for other OpenAI-compatible servers (vLLM, LM Studio, LocalAI) or remapped ports. Both llama.cpp and MLX default to port 8080, so the preset endpoint is a starting suggestion the user can edit.
  • No auto-detection: Ollama's OllamaDetector is currently dead code (no call site), so parity is presets only.
  • No native in-process MLX (mlx-swift) engine. That is a separate, large effort and not required to support MLX as a provider.
  • For llama.cpp tool calling, the server must be started with --jinja (documented).

Test plan

  • swiftlint --strict clean on changed files.
  • New unit tests cover the enum arms, registry descriptors, OpenAICompatibleProvider construction, and the default endpoint resolving to /v1/chat/completions.
  • Manual: start llama-server / mlx_lm.server, add the provider, Test Connection, fetch models, chat.

https://claude.ai/code/session_01FYwKEwKydeMG6WAHucMdY9

@mintlify

mintlify Bot commented Jul 21, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
TablePro 🟡 Building Jul 21, 2026, 4:42 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@datlechin
datlechin merged commit 25fa9f2 into main Jul 21, 2026
3 checks passed
@datlechin
datlechin deleted the feat/local-ai-providers-llamacpp-mlx branch July 21, 2026 16:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76b655bf6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +76 to +77
case .llamaCpp: return .none
case .mlx: return .none

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Expose connection testing for the unauthenticated presets

When a user adds either new preset, this .none auth style makes AIProviderDetailSheet.authSection render EmptyView; the only existing Test Connection controls are inside the API-key/Cursor/xAI auth sections. Thus llama.cpp and MLX have no way to run the connection test that the new provider instructions explicitly tell users to perform, even after choosing a model. Add an unauthenticated test control (for example in the connection section) or remove that workflow from the provider documentation.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Support for llama.cpp and Apple MLX

1 participant