feat(ai-providers): add llama.cpp and MLX as local providers#1938
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 76b655bf6d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| case .llamaCpp: return .none | ||
| case .mlx: return .none |
There was a problem hiding this comment.
Expose connection testing for the unauthenticated presets
When a user adds either new preset, this .none auth style makes AIProviderDetailSheet.authSection render EmptyView; the only existing Test Connection controls are inside the API-key/Cursor/xAI auth sections. Thus llama.cpp and MLX have no way to run the connection test that the new provider instructions explicitly tell users to perform, even after choosing a model. Add an unauthenticated test control (for example in the connection section) or remove that workflow from the provider documentation.
Useful? React with 👍 / 👎.
Closes #1777.
What
Adds llama.cpp and Apple MLX as local AI providers, alongside the existing Ollama and custom OpenAI-compatible options.
Both
llama-server(llama.cpp) andmlx_lm.server(Apple'smlx-lm) expose OpenAI-compatible HTTP servers (/v1/chat/completions,/v1/models, SSE streaming, no auth by default). TablePro'sOpenAICompatibleProvideralready drives exactly that contract, so this is an additive registry change, not new transport code. Ollama is actually the special-cased provider (native/api/chat,/api/tags, newline-JSON streaming); llama.cpp and MLX are more purely OpenAI-compatible and ride the standard path.Changes
AIModels.swift: newAIProviderTypecases.llamaCppand.mlx(display namesllama.cpp/MLX, default endpointhttp://localhost:8080, no API key, distinct SF Symbols).AIProviderRegistration.swift: register both in the OpenAI-compatible loop, so they appear in the Add Provider menu with endpoint config and model fetching.AISettingsView.swift: generalize the "Local" status label from Ollama-only to any local provider with no auth.LocalProviderRegistrationTestsmirroring the xAI registration test.Design notes
OllamaDetectoris currently dead code (no call site), so parity is presets only.--jinja(documented).Test plan
swiftlint --strictclean on changed files.OpenAICompatibleProviderconstruction, and the default endpoint resolving to/v1/chat/completions.llama-server/mlx_lm.server, add the provider, Test Connection, fetch models, chat.https://claude.ai/code/session_01FYwKEwKydeMG6WAHucMdY9