gpty-gdext

Godot 4 GDExtension that bridges the Rust terminal engine to Godot’s renderer.

GptyTerminal is a headless Node2D bridge owning the Rust grid; the visible terminal is the Control-based renderer godot/scenes/terminal/terminal_pane.gd, which polls get_grid_updates_packed() and draws in _draw().

Building

# From the workspace root
cargo build -p gpty-gdext && cd godot && godot -e

gpty.gdextension always loads res://bin/libgpty_gdext.linux.x86_64.so. For editor work, that file should be a symlink to target/debug/libgpty_gdext.so so cargo build -p gpty-gdext is enough. Restart Godot after each rebuild. ./scripts/build replaces the symlink with a release copy for export.

GDScript API

The implementation is filed by facet under src/, one #[godot_api(secondary)] impl block per file - lib.rs holds the class itself (session lifecycle, input, the grid access helpers), grid_api.rs, history_api.rs, vault_api.rs, capture_api.rs and bridge_api.rs hold the rest. The sections below are those facets.

GptyMarkdown (extends RefCounted)

Raw BBCode and HTML are escaped, images render as alt text, and only http, https, and mailto links become metadata. Godot asks for confirmation before MarkdownView opens an external link; a gpty-note: mention is emitted by this renderer alone under wikilinks, and the view hands it to its owning pane (never to OS.shell_open).

The output never uses Godot’s list tags: [ul]/[ol] lay a document out superlinearly in the item count (measured: 38915 ms to shape 1.7 MiB of two-item lists against 819 ms for that same rendered text with the tags stripped, and 146705 ms for a real 1 MiB note), so each item is a • /N. line inside one [indent] per list - identical on screen and linear to lay out. The numbering is the renderer’s own, which is also why an ordered list that starts at 3 renders as 3. rather than 1..

The input cap (MAX_MARKDOWN_BYTES) is the same 1 MiB the panes read under, so a document at a pane’s read cap renders whole, and a source past it - a pasted buffer, a routed capture, a long model turn - is what the answer’s truncated reports. MarkdownView shows that, and a cut its caller made (set_source_notice), as one banner above the body.

GptyAi (extends Node)

One private Inspector session. There is no process-global subscriber bus.

Inspector omp is launched as omp --mode rpc --no-session --no-tools --no-extensions --no-skills --no-rules.

GptyCliView (extends Node)

One cli_view pane’s CLI process: a program plus argv, never a shell. The child’s stdout and stderr are merged into one bounded queue of complete lines, which the pane drains once a frame and renders as plain text. The node strips the workspace credentials (GPTY_SECRET, GPTY_SOCKET, …) from the child’s environment, resolves a bare command through PATH and holds an absolute one to validate_executable, and kills the child and its process group on stop and on drop.

GptyMedia (extends Node)

One media pane’s decode session, backed by gpty-media and a pinned FFmpeg build. All decode work happens on the session’s own OS thread; these methods only hand over JSON or drain a bounded queue.

Errors use stable code values (bad_path, unsupported_container, unsupported_codec, too_large, open_failed, decode_failed, seek_failed, closed, io_error, plus bad_request/panic at the JSON edge). Media files are untrusted input: the backend re-validates the path, checks the demuxer and the selected codec id, caps dimensions/channels/queues, and compiles no network protocol.

GptyTerminal (extends Node2D)

Terminal lifecycle

App metadata

Vault index

Pane API & status

Grid & rendering

Scrollback & search

Selection

The selection is held by the grid (alacritty_terminal::Term::selection), not by the viewport: scrolling moves the highlight with its text, output rotates it with the grid, and the text read back for a copy includes rows scrolled out of view.

Concept engine

IPC bridge (static methods)

The control server registers version/shutdown locally, and the vault’s vaultSearch/vaultRead natively in vault_ipc.rs (a vault is a directory, not workspace state, and the walk plus SQLite run on the blocking pool instead of the GUI thread); the vault’s declared-weight write (vaultWeight) is the exception - its store is Godot’s user://link_weights.json, so it is routed to GDScript like every other stateful method. Every other method (newPane, paneRead, paneStatus, paneRun, paneWait, broadcast, layout, concepts) is listed in ipc.rs gdscript_methods and routed to GDScript - workspace.gd delegates dispatch to WorkspaceIpcHandlers (ipc_handlers.gd). New pane-API methods must be added to both the registration list and the handler module, unless they are directory-scoped like the vault pair.

Grid Update Dictionary

get_grid_updates_packed() returns one of two shapes. Both carry flat per-cell arrays: fg/bg are PackedColorArray, attrs is PackedInt32Array (bit flags: 1 bold, 2 italic, 4 underline, 8 inverse, 16 wide), and chars is an Array of strings.

fg/bg are paint-ready: CellInfo::paint_colors has already applied SGR 7 (inverse), so fg is always the glyph color and bg always the background. Draw them as they are - do not swap an inverse cell in GDScript. The 8 inverse bit stays in attrs as cell information, not as a drawing instruction; swapping only the text color is what painted a bracketed paste (readline echoes it inside ESC[7m) in its own background.

Full update (is_full = true, first fetch or force_full):

{
    "is_full": true,
    "rows": 24, "cols": 80,          # int - grid dimensions
    "chars": ["line0", "line1", …],  # Array[String] - one string per row
    "fg": PackedColorArray(…),       # len = rows × cols
    "bg": PackedColorArray(…),
    "attrs": PackedInt32Array(…),
}

Partial update (is_full = false, damaged cells only):

{
    "is_full": false,
    "indices": PackedInt32Array(…),  # cell index = row * cols + col
    "chars": ["A", "B", …],          # Array[String] - one string per cell
    "fg": PackedColorArray(…),
    "bg": PackedColorArray(…),
    "attrs": PackedInt32Array(…),
}

OMP event socket (Reasoning pane)

Passive OMP observability uses a second local IPC listener beside the workspace-control socket: gpty.sock → gpty-events.sock on Unix, \\.\pipe\gpty → \\.\pipe\gpty-events on Windows. Implementation: src/omp_events.rs. Protocol details: crates/gpty-ipc/README.md.

The listener runs on every platform, and it is started unconditionally: every PTY spawn calls ensure_server_started() and register_terminal() - no cfg branch stands between the two - and the shell is given GPTY_EVENT_* when registration succeeds. @gpty/omp-events then forwards semantic events and workspace.gd polls drain_agent_events() into the Reasoning pane, so Terminal, Inspector and Reasoning behave the same way on Windows. The windows-smoke job runs the extension inside a real pane there and asserts the pane reports Tier 1 working, which observes the whole chain - env injection, transport, capability check, translation, drain - on the platform.

The shipped extension needs no platform branch to reach either transport: net.connect(path) opens a Unix socket for a socket path and a named pipe for a pipe path. The only branch is validation - a Windows path must be a \\.\pipe\ name, answered from the path alone because a pipe cannot be stated, while on Unix the socket must be owned by this user and inaccessible to group/other.

The event code carries no #[cfg(unix)] gates at all: omp_events.rs is platform-neutral, which is what lets the rust-windows job compile this crate under RUSTFLAGS: -Dwarnings.

Tips

• Use get_grid_generation() to skip redundant grid polls when idle
• The renderer uses get_grid_updates_packed() to fetch only damaged cells, merging into _cell_cache
• If the grid mutex is held by the background task, or no shell has started yet, get_grid_updates_packed() returns an empty Dictionary and get_rows() / get_cols() answer 0