linuxserver · thelamer · Jul 30, 2026
diff --git a/docs/selkies/.pages b/docs/selkies/.pages
@@ -0,0 +1,5 @@
+nav:
+    - Overview: index.md
+    - User Guide: user-guide
+    - Developer Guide: developer-guide
+    - Components: components
diff --git a/docs/selkies/components/.pages b/docs/selkies/components/.pages
@@ -0,0 +1,9 @@
+nav:
+    - index.md
+    - Selkies: selkies.md
+    - Pixelflux and Pcmflux: pixelflux.md
+    - Selkies Baseimages: baseimages.md
+    - Selkies Desktop: selkies-desktop.md
+    - SealSkin: sealskin.md
+    - SealSkin Apps Registry: sealskin-apps.md
+    - Pelorus: pelorus.md
diff --git a/docs/selkies/components/baseimages.md b/docs/selkies/components/baseimages.md
@@ -0,0 +1,75 @@
+# Selkies Baseimages
+
+**Repository:** [linuxserver/docker-baseimage-selkies](https://github.com/linuxserver/docker-baseimage-selkies) · **Registries:** `ghcr.io/linuxserver/baseimage-selkies`, `lsiobase/selkies`
+
+`docker-baseimage-selkies` is the packaging layer of the platform, the image every Webtop and single application container is built FROM. It assembles the whole runtime, the compositor stack, Selkies, pixelflux and pcmflux, Nginx, PulseAudio, GPU detection, hardening, and the LinuxServer.io s6 service machinery, so that a downstream image only has to install an application and say how to start it.
+
+This page is the component overview. The [Developer Guide](../developer-guide/index.md) covers building on it ([Building Custom Images](../developer-guide/building-images.md)) and its internals in depth ([Baseimage Internals](../developer-guide/baseimage-internals.md)).
+
+## Available distros
+
+One branch and tag per base distribution, all for x86_64 and aarch64:
+
+| Distro | Tag |
+| --- | --- |
+| Alpine | `alpine324` |
+| Arch | `arch` |
+| Debian | `debiantrixie` |
+| Fedora | `fedora44` |
+| Kali | `kali` |
+| Ubuntu | `ubunturesolute` |
+
+There is deliberately **no `latest` tag** for base images. Downstream images pin a distro tag, for example `FROM ghcr.io/linuxserver/baseimage-selkies:debiantrixie`.
+
+## What is inside
+
+- **The LSIO foundation**: each tag builds on the corresponding LinuxServer.io distro baseimage, inheriting s6-overlay init, the `abc` user with PUID and PGID remapping, `TZ`, Docker mods, and the `/custom-cont-init.d` and `/custom-services.d` hooks.
+- **Selkies** (pinned commit) installed into the `/lsiopy` virtualenv, with pixelflux 2.x and pcmflux 2.x, plus [Pelorus](pelorus.md) preinstalled.
+- **The web client**: prebuilt dashboards under `/usr/share/selkies/`, selected at runtime by the `DASHBOARD` variable.
+- **Compositors for both stacks**: labwc (built from source with a small IPC patch that adds the window query socket Pelorus uses) for Wayland, and a patched Xvfb (with `-vfbdevice` DRI3 support) plus Openbox for the legacy X11 fallback. A patched wlroots build makes the compositor survive pixman rendering faults instead of crashing.
+- **[Selkies Desktop](selkies-desktop.md)** at `/usr/bin/selkies-desktop`, activated by env var.
+- **Nginx** with the fancyindex module, serving the client, proxying the WebSocket, handling basic auth, subfolder support, and the `/files` download index.
+- **PulseAudio** with null sinks (`output` and `input`) wired for stream audio and microphone return.
+- **Gamepad plumbing**: the joystick interposer and fake udev libraries, preloaded globally, with device nodes created at init.
+- **Quality of life**: passwordless sudo for the desktop user, all system locales prebuilt for `LC_ALL`, `proot-apps` synced into the user home for persistent app installs, Docker in Docker support for privileged containers, and notification support.
+
+## The runtime in one diagram
+
+```mermaid
+graph TD
+    subgraph s6 services
+        NGINX[svc-nginx]
+        PULSE[svc-pulseaudio]
+        SELKIES[svc-selkies: the selkies server]
+        DE[svc-de: startwm script]
+        WD[svc-watchdog: RESTART_APP]
+    end
+    SELKIES -->|starts in process| PF[pixelflux Wayland compositor, socket wayland-1]
+    DE -->|waits for wayland-1| LABWC[labwc or a full DE, exposes wayland-0]
+    LABWC --> APP[autostart application]
+    NGINX -->|3000 / 3001| WEB[web client, /websocket proxy, /files, /pelorus]
+```
+
+At startup a chain of one shot init scripts configures everything from environment variables: Nginx substitution (ports, auth, subfolder, title), Wayland or X11 mode selection, first run copy of the autostart and menu defaults into `/config`, hardening (the `HARDEN_*` and `DISABLE_*` family), GPU detection and permission fixes, and gamepad device setup. Then the long running services above come up in dependency order.
+
+## The two session modes
+
+- **Wayland (default on capable hardware)**: pixelflux hosts the virtual compositor; labwc (single apps) or a full desktop (Webtop flavors) nests on it; zero copy GPU encoding is available. `PIXELFLUX_WAYLAND=true` is baked into current downstream images.
+- **X11 (legacy fallback)**: patched Xvfb with DRI3, Openbox, XSHM capture. Selected with `PIXELFLUX_WAYLAND=false` or on flavors that have not moved to Wayland yet. Deprecated for GPU work.
+
+## The downstream contract
+
+A downstream image customizes the base by providing a handful of well known files:
+
+| File | Purpose |
+| --- | --- |
+| `/defaults/autostart_wayland` and `/defaults/autostart` | The command that launches your app (Wayland and X11 variants) |
+| `/defaults/menu_wayland.xml` and `/defaults/menu.xml` | The right click root menu |
+| `/defaults/startwm_wayland.sh` and `/defaults/startwm.sh` | Replace the whole session for full desktop images |
+| `/usr/share/selkies/www/icon.png` | The app icon used for the PWA and favicon |
+
+Plus `ENV TITLE`, and whatever packages the app needs. That is the entire interface, [Building Custom Images](../developer-guide/building-images.md) walks through real examples.
+
+## Versioning and builds
+
+Images rebuild on a weekly package check cadence and on baseimage changes, through the standard LinuxServer.io Jenkins pipeline. Each downstream repository (chromium, webtop, and the rest of the fleet) is triggered from its own upstream signal, app version bumps or OS package updates, so the whole catalog stays current without manual intervention.
diff --git a/docs/selkies/components/index.md b/docs/selkies/components/index.md
@@ -0,0 +1,66 @@
+# Components
+
+The platform is built from focused subprojects, each independently useful, each documented here with what it is, how it works, and where it sits in the stack.
+
+## The map
+
+```mermaid
+graph TD
+    subgraph Clients
+        BR[Web browser client]
+        EXT[SealSkin extension and mobile apps]
+        AI[LLM agents]
+    end
+    subgraph Orchestration
+        SS[SealSkin server]
+        REG[SealSkin apps registry]
+    end
+    subgraph "App containers (one per session)"
+        NG[Nginx]
+        SEL[Selkies server]
+        PF[Pixelflux video]
+        PCM[Pcmflux audio]
+        PEL[Pelorus]
+        WM[labwc or KDE Plasma]
+        SD[Selkies Desktop shell]
+        APP[Application]
+    end
+    BI[docker-baseimage-selkies] -.packages everything above the line.-> NG
+    EXT --> SS
+    SS --> REG
+    SS -->|launches| NG
+    BR --> NG
+    AI --> PEL
+    NG --> SEL
+    SEL --> PF
+    SEL --> PCM
+    PF --> WM
+    WM --> APP
+    SD -.optional shell.-> WM
+    PEL -.optional agent API.-> WM
+```
+
+## The projects
+
+| Component | One line | Layer |
+| --- | --- | --- |
+| [Selkies](selkies.md) | The streaming server and web client: WebSocket protocol, input, audio, clipboard, files, sharing | Session server |
+| [Pixelflux and Pcmflux](pixelflux.md) | Capture and encoding engines for video (including the in process Wayland compositor) and audio | Media pipeline |
+| [Selkies Baseimages](baseimages.md) | The OCI baseimages that package the whole stack with s6, Nginx, GPU detection, and LSIO conventions | Packaging |
+| [Selkies Desktop](selkies-desktop.md) | A tiny panel, start menu, taskbar, and desktop icon shell for single app containers | Optional shell |
+| [SealSkin](sealskin.md) | Self hosted orchestration: users, auth, on demand container launching, extensions, and mobile apps | Orchestration |
+| [SealSkin Apps Registry](sealskin-apps.md) | The YAML app manifests and autostart scripts that define the launchable catalog | Orchestration data |
+| [Pelorus](pelorus.md) | Agentic interface: text based desktop state and a control API so LLMs can drive sessions | Agent layer |
+
+## How a frame reaches your eyeball
+
+To make the layering concrete, here is the life of one frame in a Wayland mode container:
+
+1. The application renders into a buffer belonging to **labwc** (or KWin on KDE), which is itself a client of the headless Smithay compositor that **pixelflux** hosts in process.
+2. Pixelflux composites the output. If a GPU holds the framebuffer and the encoder lives on the same GPU, the frame is passed as a DMA-BUF straight into NVENC or VA-API, zero copy. Otherwise it is read back and encoded on CPU, in parallel stripes if the software encoder is in use.
+3. Only regions that changed get encoded at all; a static screen costs almost nothing, and after motion stops a high quality paint over pass restores perfect detail.
+4. The encoded frame, with a small binary header, is handed to **Selkies**, which broadcasts it over the WebSocket to every connected viewer with backpressure control per client.
+5. The container's **Nginx** carries that WebSocket alongside the static web client, file downloads, basic auth, and the optional Pelorus API, all on one HTTPS port.
+6. In your browser, the **web client** decodes with WebCodecs and paints to a canvas, while sending your input, clipboard, mic, and gamepad state back up the same socket.
+
+Everything above ships in one container image built on **docker-baseimage-selkies**, and **SealSkin** launches, secures, and reaps such containers on demand.
diff --git a/docs/selkies/components/pelorus.md b/docs/selkies/components/pelorus.md
@@ -0,0 +1,81 @@
+# Pelorus
+
+**Repository:** [linuxserver/pelorus](https://github.com/linuxserver/pelorus) · **Enable with:** `-e PELORUS=true` · **UI at:** `https://<host>:3001/pelorus/`
+
+Pelorus is the agentic interface for Selkies desktops: an AI navigator that lets a chat LLM drive a real Linux desktop. It works with labwc sessions (single application containers) and full KDE Plasma desktops, and it is preinstalled in every Selkies baseimage, activated with a single environment variable.
+
+The key idea that separates Pelorus from screenshot driven "computer use" agents: **the desktop is represented as text.** Window lists with exact geometry, accessibility trees with precise click coordinates for every button and menu item, taskbar and start menu contents, all delivered as structured text. A model never has to guess pixel coordinates from an image, which means even small quantized local models can operate the desktop reliably. Screenshots exist as a fallback for content that has no accessibility representation (games, canvases, some Electron apps).
+
+## Enabling it
+
+```bash
+docker run --rm -it \
+  --shm-size=1gb \
+  -p 3001:3001 \
+  -e PELORUS=true \
+  lscr.io/linuxserver/chromium:latest bash
+```
+
+`PELORUS=true` in Wayland mode does all of this automatically:
+
+- Starts the Pelorus FastAPI server (port 5100 internally, proxied by the container's Nginx at `/pelorus/`)
+- Enables the pixelflux **Computer Use API** on port 5000 internally (`PIXELFLUX_CU=5000`), the raw input injection and screenshot layer
+- Starts the AT-SPI accessibility registry and forces the accessibility bridge on for GTK and Qt applications so their UI trees are readable
+- On labwc, starts the compositor with its IPC socket (`labwc -i`) so windows can be enumerated; on KDE, window data comes from KWin over D-Bus
+
+Open `https://<host>:3001/pelorus/` for the built in chat UI, or `.../pelorus/docs` for the OpenAPI reference.
+
+## How the agent loop works
+
+1. **Observe:** capture desktop state as text, the environment header, desktop icons, taskbar buttons, one line per window with PID, title, and geometry, plus any open start menu contents.
+2. **Infer:** send state, system prompt, and the task to the configured model. Supported providers: **Ollama**, any **OpenAI compatible** endpoint, and **Gemini**.
+3. **Act:** dispatch the model's tool calls, input actions go to the Computer Use backend, introspection actions are answered locally.
+4. Repeat until the task completes or the step budget runs out.
+
+The model gets three tools:
+
+- **`computer`**: the workhorse. Actions include `desktop_state`, `explore_window` (dump a window's full accessibility tree with exact screen coordinates for every element), `key`, `type`, the click family, `scroll`, `left_click_drag`, `hold_key`, `wait`, and `close_window`. `explore_window` is the primary inspection tool; a vision screenshot of the window region is the fallback when a tree is unavailable.
+- **`create_task`** and **`set_task_status`**: a small task queue so the agent can decompose work.
+
+## Using Pelorus from your own code
+
+You do not have to use the built in agent. The REST API exposes the desktop primitives directly, so an external agent (or your own orchestration) can be the brain. The full API reference with request and response schemas lives in [API.md in the Pelorus repository](https://github.com/linuxserver/pelorus/blob/master/API.md); the highlights:
+
+| Endpoint | Purpose |
+| --- | --- |
+| `POST /api/run` | Run the built in agent on a task, optionally streaming steps over SSE |
+| `GET /api/state` | The text based desktop state |
+| `GET /api/windows` | Window list with geometry |
+| `GET /api/desktop/explore/{pid}` | Accessibility tree for a window, coordinates enriched to absolute screen positions |
+| `POST /api/desktop/control` | Raw input: clicks, typing, keys, scrolling, drags, screenshots, region zoom |
+| `GET /api/desktop/screenshot` | Full desktop screenshot, base64 PNG |
+| `POST /api/desktop/close/{pid}` | Close a window |
+| `WS /ws` | Streaming agent protocol used by the chat UI |
+
+Provider configuration lives at `/config/agent/config.toml` (auto created, seeded from `PELORUS_PROVIDER`, `PELORUS_ENDPOINT`, `PELORUS_MODEL`, and `PELORUS_API_KEY` environment variables), and can be managed at runtime through `/api/servers`.
+
+## Configuration reference
+
+| Variable | Default | Description |
+| --- | --- | --- |
+| `PELORUS` | unset | Set `true` in a baseimage container to enable the whole stack |
+| `PELORUS_PORT` | `5100` | Internal API port |
+| `PELORUS_PROVIDER` | `ollama` | `ollama`, an OpenAI compatible provider, or `gemini` |
+| `PELORUS_ENDPOINT` | `http://localhost:11434` | Model endpoint URL |
+| `PELORUS_MODEL` | `gemma4:12b` | Model name |
+| `PELORUS_API_KEY` | empty | API key where the provider needs one |
+| `PIXELFLUX_CU` | `5000` when `PELORUS=true` | Computer Use API port, set automatically |
+
+## Security
+
+Treat Pelorus as an unauthenticated root of control over the session:
+
+- The API has **no authentication of its own**. In the containers it is reachable only through Nginx (which can carry the container's basic auth) and is not published directly. Never expose port 5100 or 5000 yourself.
+- Configured provider API keys are stored in `/config/agent/config.toml` and are readable through the server management API, anyone who can reach the API can read them.
+- Enabling the accessibility bridge means every process in the session can read the full UI text of every application. That is inherent to how AT-SPI works and is the price of text based desktop state.
+
+The [Security and Hardening](../user-guide/security.md) page covers the container level mitigations.
+
+## How it fits the stack
+
+Pelorus is a consumer of two lower layers documented elsewhere: the pixelflux [Computer Use API](pixelflux.md) for input and pixels, and the compositor's window enumeration (the labwc IPC patch shipped in the [baseimages](baseimages.md), or KWin D-Bus on KDE). It adds the accessibility layer, the state formatting, the LLM providers, and the agent loop on top. If you are building your own agentic system, that layering is the map: you can plug in at any level.