Skip to content

Testing

  • UI/UX assessment method - used by the v0.8.0 and v0.8.1 re-baselines. Run the real app (the actual main scene, autoloads and Rust extension) in a private sandbox with a fresh first-run state, so no existing settings or layouts influence it; drive it with synthetic clicks and key events from a scratch script and save the app’s own rendered viewport as PNG - never a desktop capture; read the UI code to explain what the captures show and to count hard-coded styling; compute contrast as WCAG ratios from colour constants in the code or from sampled pixels; render small windows (800×500, 560×340) in an embedded viewport, which is accurate for layout but not for OS window behaviour. Mark each finding verified (reproduced in the running app, with a control where one mattered), observed (seen once or without a control) or code (read in the source, not exercised); roadmap rows use these words, plus not tested and measured. List what the assessment did not examine, and queue that list as a row instead of dropping it. Do not modify project files while assessing.
  • Tests live in godot/tests/ - unit/ for pure-logic classes, integration/ for scene-tree tests. A test that needs a pane’s shell sets SettingsManager.cfg_shell_command = SettingsManager.default_shell_command() (never a literal /bin/sh: on Windows that is refused by validate_executable and the spawn fails as an engine error, which GUT counts as a test failure), and a test that needs exact bytes from a child builds its command line with ShellFixtures.print_text() (godot/tests/helpers/shell_fixtures.gd) - and a test that needs a mode to come from the child rather than from its shell spawns its pane with ShellFixtures.no_line_editing_args() (bash --noediting; the default shell sets DECSET 2004 around every line its readline edits, which made test_bracketed_paste.gd pass 3/3 even with the fixture’s command line replaced by a bare sleep 30) - spawning through Workspace._spawn_pane, not TerminalManager.spawn_pane, because only the former parents the wrapper so _ready runs the shell - and a transition (on, then off) in that file uses ShellFixtures.print_text_pair(), since a holding fixture cannot be typed into a second time - or, when its child is spawned as argv instead of typed into a shell (a cli_view pane), with ShellFixtures.print_text_argv(): print_text() returns a cmd.exe command line, and on Windows argv hands that line to cmd.exe /c, which re-parses it - \" is not cmd syntax, so PowerShell received the script as a quoted string and printed it rather than running it (measured on windows-smoke), which is why the argv form names PowerShell directly there and keeps the shell on POSIX - and, when it waits for those bytes, waits for the shell to be printing first (a nonzero idle_ms in get_status(): a line typed into a console app that has not attached yet is read by nothing) and then gives the child’s own startup a budget sized for a process launch rather than for the assertion. A cold powershell.exe on a Windows runner measured past the 10 s test_bracketed_paste.gd used to allow, which reported a cold start as “the mode never arrived” (windows-smoke).
  • Mocking autoloads: Use MockAutoloads.setup() / teardown() in before_each/after_each.
  • Persistence managers (SettingsManager, ProfileManager, WorkspaceStore) are mocked via set_script() on the existing autoload node, redirecting _read_file/_write_file to an in-memory Dictionary - avoids touching disk and preserves Godot 4 global name bindings. (SettingsManager etc. are static constants - never free() autoload nodes).
  • Signal testing: GDScript lambdas cannot capture outer primitives. Use GUT’s watch_signals(node) + assert_signal_emitted(node, "signal_name") instead of node.signal.connect(func(): captured_var = true).
  • Editor tests must type, not assign: a TextEdit/CodeEdit programmatic text = … goes through _set_text(p_text, /*emit_signal=*/false) and emits only text_set - text_changed is emitted for edits (insert_text_at_caret, remove_text, undo) and is deferred, one frame later (call_deferred; Godot 4.7.2, scene/gui/text_edit.cpp). So a test that assigned text and asserted a live preview or a dirty marker would assert nothing, and code that trusts the signal’s timing would break on both counts - wiki_pane.gd recomputes its dirty state from the buffer instead, and test_wiki_pane.gd’s _type_into() driver (select_all + insert_text_at_caret + one process frame) is the way edits reach it in a test.
  • Type checks: body is SomeClass requires a compile-time class name. For runtime type discrimination, use body._pane_type() string discriminators.
  • Headless resource leaks: GUT warnings about unfreed children and GDExtension RID/ObjectDB leaks are benign in headless mode - the dummy render server doesn’t track GDExtension resources. Production renderer handles these correctly.
  • A new class_name needs godot --headless --path godot --import before gut-check can see it: headless runs resolve global classes from godot/.godot/global_script_class_cache.cfg, which only the importer refreshes (that is why ci-check imports before GUT) - running gut-check alone right after adding a class fails with “Could not find type”.
  • The GUT gate is ./scripts/gut-check, never the bare godot -s addons/gut/gut_cmdln.gd command (ci-check and ci.yml both call the script; CONTRIBUTING and the testing docs point at it). GUT exits 0 after a parse error aborts the run - measured 4 of 40 scripts, 1.3 s instead of 57 s, no Run Summary - so the script owns the command and judges the output instead: Godot parse/script-error lines fail it, the Run Summary must be present and end in All tests passed!, the reported script count must equal the test_*.gd files the gdirs hold, and every script must contribute at least one test. Adding a test directory means adding it to that script, not just to a command line: GUT does not recurse (include_subdirectories is off), and a test_*.gd nested below a gdir is reported as never-run rather than quietly skipped. The script also owns a private endpoint and profile for the run, which is not hygiene but the gate working at all: the suite spawns real panes, every instance binds the control and event endpoints, and one instance owns each - so a developer’s open workspace used to make every pane-spawning test refuse its bind, set the test instance’s shutdown flag and quit it mid-suite (measured 2026-10-14: 12 scripts, no Run Summary, which reads as a crash rather than as “close the app”). GPTY_SOCKET moves both listeners at once (the event path is derived from the control one; on Windows it is the pipe name), XDG_RUNTIME_DIR/XDG_DATA_HOME/XDG_STATE_HOME point at one 0700 temp dir the trap removes, and GPTY_SECRET is cleared so a developer’s shell cannot make the test instance demand a secret no test client has. The suite’s user root is the same sandbox: GPTY_TEST_USER_DIR is set on every platform, and it is the one seam both halves read - BasePersistenceManager.resolve_store_path redirects every user:// store through it and the gdext history_db_path redirects the scrollback database the same way. Setting it on Unix too is deliberate (the XDG dirs already cover Godot’s user:// there): the normal Linux gate exercises the seam, so a Windows-only path cannot drift unnoticed. On Windows, where the XDG variables do nothing because Godot resolves user:// through the Known Folder API, LOCALAPPDATA/APPDATA are sandboxed as well (the Rust state/data directories read them directly). The GUT run does not use --log-file: Godot’s file logging is off by default (debug/file_logging is unset), so there is no engine log to move, and the tee capture already holds stdout for the gate’s judgment. The smoke does use it (it needs a failure-time engine log); adding it to the GUT command was the only difference from the known-good Windows run and the step hung with both the MSYS and cygpath spellings, so it stays out until a Windows run pins that down. GUT_TIMEOUT (600 s) caps the Godot process so a hung suite fails the gate instead of holding the runner to the six-hour job cap (stock macOS has no timeout, where the gate runs unbounded as before). -d is not passed: Godot parses it as --debug, and a runtime error in a test then breaks into the interactive debugger - measured on Windows as a 1.7 GB flood of the same Debugger Break that stalled the suite until the timeout; without it GUT records the failure and the gate’s own checks fail the run. Tests that need a filesystem path use TestPaths.user_path(...), never ProjectSettings.globalize_path("user://..."): a bare user:// path bypasses the seam and on Windows writes the developer’s real profile. Do not add a second rule - BasePersistenceManager.resolve_store_path is the store half, history_db_path the gdext half, and TestPaths.user_dir() asks the same user_root_override() for the directory.
  • The live pane-API smoke is scripts/smoke_pane_api.py - one harness for every platform, because the transport is the only difference between them (Unix socket vs named pipe; the event client is the same file over open()). ci-check and the Linux CI job run it with python3; the windows-smoke job on windows-latest runs the same file with python, which is the only place the Windows named-pipe control/event listeners and a real Windows pane (default shell %COMSPEC%) are exercised. GUT runs on both: the suite starts panes with SettingsManager.default_shell_command() and asks ShellFixtures.print_text() for a command line that emits exact bytes on either platform (POSIX printf '%b', Windows PowerShell [Console]::Write), so the windows-smoke job runs scripts/gut-check too (under Git Bash - the gate is one bash script for both platforms). Windows gaps pend rather than fail, and each pend names its gap (a fake adapter written as a POSIX shell script; a world-writable file mode, which is not a Windows notion; the three gpty_state declaration tests, where ConPTY consumes the OSC): gut-check prints every pending test and fails on [Risky] (a test that ran without asserting), so an explicitly-pended platform gap cannot hide a suite that stopped asserting. Pending is a Windows-only allowance: outside Windows gut-check fails on any [Pending] test, because there it can only mean a skip nobody decided on (the extension is a build requirement, so a test that needs GptyAi or GptyCliView asserts it is present instead of standing down). A step that waits for a program the pane has to launch sizes its budget for that launch, not for the round trip the step is about: node outran the 10 s the extension step used to allow on windows-smoke (the pane tail held the echoed command line and nothing else, idle_ms ~9.8 s), so the step reported “the event never arrived” for a Node.js that was still starting - the same lesson as the GUT fixture’s cold powershell.exe. The cli_view step runs in the same harness: it creates a pane through new-pane --type cli_view --command <program> --arg … and asserts the child’s stdout arrives in pane-read, which is what proves the argv spawn surface live on whichever platform runs the smoke. The concept step seeds one concept into the run’s sandboxed user root before launch - the only registration route that exists, since concept toggle flips a shipped rule rather than adding one; the concept triggers on a marker the shell builds at runtime (the trigger demands the appended digit, so no typed line can match it), and the step asserts the routed {type: concept, name, target} event on the event socket. That is the Windows proof for the ConPTY parser fix: the event can only come from a committed output line, which is exactly what ConPTY’s cursor-move line breaks used to suppress. The final step seeds a plugin record into the sandboxed state store and asserts layout list offers the plugin’s profile and layout load restores its tile, which is the live proof of the profiles migration’s read path: store → installed_plugin_profiles() → ProfileManager → layoutList/layoutLoad. Every one of those steps is also a CLI teardown sample, and the pane-status stage adds one deliberately: 100 throwaway pane-status processes (≈ 200 connections - each process opens the best-effort version probe and then its command), the amplifier for the one Windows STATUS_HEAP_CORRUPTION (0xC0000374) the smoke saw on an inject; a nonzero exit fails there with its hex status, and it is never retried.
  • Media backend tests (crates/gpty-media/tests): committed fixtures generated from FFmpeg’s lavfi sources cover every allowed container/codec (H.264/HEVC/VP9/AV1/Theora video; Vorbis/Opus/FLAC/MP3/AAC/PCM audio). They assert probe metadata, packed RGBA/PCM output at the requested rate/channels, seek landing at or after the target, the path gate, clean refusal of garbage/truncated files, close-while-queued (no deadlock), and the two seam checks - build_recipe.rs (components.txt ↔ runtime allowlists) and binding_rules.rs (banned FFmpeg APIs). They link the pinned FFmpeg: run ./scripts/media-deps first (every CI job that compiles or loads the extension does).
  • Seam contract tests: a contract written in one module and implemented in another needs a test that reads both sides, never a copy of the other side written into the test - the copy passes while the shipped code drifts. crates/gpty-cli/src/commands/mcp.rs parses the real IPC registrations (ipc.rs), the real dispatch (ipc_handlers.gd arms plus the method == requests workspace.gd intercepts) and every client.call literal under src/commands/, and asserts they agree with the advertised MCP tool names; godot/tests/integration/test_ipc_dispatch_contract.gd drives WorkspaceIpcHandlers.handle against a real Workspace (one instance for the script: each schedules a deferred concept push that must land while it is alive) instead of reimplementing the handlers, and scripts/smoke_pane_api.py proves pane-read’s scrollback claim live, where output timing is real.