Testing
- UI/UX assessment method - used by the v0.8.0 and v0.8.1 re-baselines. Run the real app (the actual main scene, autoloads and Rust extension) in a private sandbox with a fresh first-run state, so no existing settings or layouts influence it; drive it with synthetic clicks and key events from a scratch script and save the app’s own rendered viewport as PNG - never a desktop capture; read the UI code to explain what the captures show and to count hard-coded styling; compute contrast as WCAG ratios from colour constants in the code or from sampled pixels; render small windows (800×500, 560×340) in an embedded viewport, which is accurate for layout but not for OS window behaviour. Mark each finding verified (reproduced in the running app, with a control where one mattered), observed (seen once or without a control) or code (read in the source, not exercised); roadmap rows use these words, plus not tested and measured. List what the assessment did not examine, and queue that list as a row instead of dropping it. Do not modify project files while assessing.
- Tests live in
godot/tests/-unit/for pure-logic classes,integration/for scene-tree tests. A test that needs a pane’s shell setsSettingsManager.cfg_shell_command = SettingsManager.default_shell_command()(never a literal/bin/sh: on Windows that is refused byvalidate_executableand the spawn fails as an engine error, which GUT counts as a test failure), and a test that needs exact bytes from a child builds its command line withShellFixtures.print_text()(godot/tests/helpers/shell_fixtures.gd) - and a test that needs a mode to come from the child rather than from its shell spawns its pane withShellFixtures.no_line_editing_args()(bash --noediting; the default shell sets DECSET 2004 around every line its readline edits, which madetest_bracketed_paste.gdpass 3/3 even with the fixture’s command line replaced by a baresleep 30) - spawning throughWorkspace._spawn_pane, notTerminalManager.spawn_pane, because only the former parents the wrapper so_readyruns the shell - and a transition (on, then off) in that file usesShellFixtures.print_text_pair(), since a holding fixture cannot be typed into a second time - or, when its child is spawned as argv instead of typed into a shell (acli_viewpane), withShellFixtures.print_text_argv():print_text()returns a cmd.exe command line, and on Windows argv hands that line tocmd.exe /c, which re-parses it -\"is not cmd syntax, so PowerShell received the script as a quoted string and printed it rather than running it (measured onwindows-smoke), which is why the argv form names PowerShell directly there and keeps the shell on POSIX - and, when it waits for those bytes, waits for the shell to be printing first (a nonzeroidle_msinget_status(): a line typed into a console app that has not attached yet is read by nothing) and then gives the child’s own startup a budget sized for a process launch rather than for the assertion. A coldpowershell.exeon a Windows runner measured past the 10 stest_bracketed_paste.gdused to allow, which reported a cold start as “the mode never arrived” (windows-smoke). - Mocking autoloads: Use
MockAutoloads.setup()/teardown()inbefore_each/after_each. - Persistence managers (
SettingsManager,ProfileManager,WorkspaceStore) are mocked viaset_script()on the existing autoload node, redirecting_read_file/_write_fileto an in-memory Dictionary - avoids touching disk and preserves Godot 4 global name bindings. (SettingsManageretc. are static constants - neverfree()autoload nodes). - Signal testing: GDScript lambdas cannot capture outer primitives. Use GUT’s
watch_signals(node)+assert_signal_emitted(node, "signal_name")instead ofnode.signal.connect(func(): captured_var = true). - Editor tests must type, not assign: a
TextEdit/CodeEditprogrammatictext = …goes through_set_text(p_text, /*emit_signal=*/false)and emits onlytext_set-text_changedis emitted for edits (insert_text_at_caret,remove_text, undo) and is deferred, one frame later (call_deferred; Godot 4.7.2,scene/gui/text_edit.cpp). So a test that assignedtextand asserted a live preview or a dirty marker would assert nothing, and code that trusts the signal’s timing would break on both counts -wiki_pane.gdrecomputes its dirty state from the buffer instead, andtest_wiki_pane.gd’s_type_into()driver (select_all+insert_text_at_caret+ one process frame) is the way edits reach it in a test. - Type checks:
body is SomeClassrequires a compile-time class name. For runtime type discrimination, usebody._pane_type()string discriminators. - Headless resource leaks: GUT warnings about unfreed children and GDExtension
RID/ObjectDBleaks are benign in headless mode - the dummy render server doesn’t track GDExtension resources. Production renderer handles these correctly. - A new
class_nameneedsgodot --headless --path godot --importbeforegut-checkcan see it: headless runs resolve global classes fromgodot/.godot/global_script_class_cache.cfg, which only the importer refreshes (that is whyci-checkimports before GUT) - runninggut-checkalone right after adding a class fails with “Could not find type”. - The GUT gate is
./scripts/gut-check, never the baregodot -s addons/gut/gut_cmdln.gdcommand (ci-checkandci.ymlboth call the script; CONTRIBUTING and the testing docs point at it). GUT exits 0 after a parse error aborts the run - measured 4 of 40 scripts, 1.3 s instead of 57 s, no Run Summary - so the script owns the command and judges the output instead: Godot parse/script-error lines fail it, the Run Summary must be present and end inAll tests passed!, the reported script count must equal thetest_*.gdfiles the gdirs hold, and every script must contribute at least one test. Adding a test directory means adding it to that script, not just to a command line: GUT does not recurse (include_subdirectoriesis off), and atest_*.gdnested below a gdir is reported as never-run rather than quietly skipped. The script also owns a private endpoint and profile for the run, which is not hygiene but the gate working at all: the suite spawns real panes, every instance binds the control and event endpoints, and one instance owns each - so a developer’s open workspace used to make every pane-spawning test refuse its bind, set the test instance’s shutdown flag and quit it mid-suite (measured 2026-10-14: 12 scripts, no Run Summary, which reads as a crash rather than as “close the app”).GPTY_SOCKETmoves both listeners at once (the event path is derived from the control one; on Windows it is the pipe name),XDG_RUNTIME_DIR/XDG_DATA_HOME/XDG_STATE_HOMEpoint at one 0700 temp dir the trap removes, andGPTY_SECRETis cleared so a developer’s shell cannot make the test instance demand a secret no test client has. The suite’s user root is the same sandbox:GPTY_TEST_USER_DIRis set on every platform, and it is the one seam both halves read -BasePersistenceManager.resolve_store_pathredirects everyuser://store through it and the gdexthistory_db_pathredirects the scrollback database the same way. Setting it on Unix too is deliberate (the XDG dirs already cover Godot’suser://there): the normal Linux gate exercises the seam, so a Windows-only path cannot drift unnoticed. On Windows, where the XDG variables do nothing because Godot resolvesuser://through the Known Folder API,LOCALAPPDATA/APPDATAare sandboxed as well (the Rust state/data directories read them directly). The GUT run does not use--log-file: Godot’s file logging is off by default (debug/file_loggingis unset), so there is no engine log to move, and theteecapture already holds stdout for the gate’s judgment. The smoke does use it (it needs a failure-time engine log); adding it to the GUT command was the only difference from the known-good Windows run and the step hung with both the MSYS andcygpathspellings, so it stays out until a Windows run pins that down.GUT_TIMEOUT(600 s) caps the Godot process so a hung suite fails the gate instead of holding the runner to the six-hour job cap (stock macOS has notimeout, where the gate runs unbounded as before).-dis not passed: Godot parses it as--debug, and a runtime error in a test then breaks into the interactive debugger - measured on Windows as a 1.7 GB flood of the sameDebugger Breakthat stalled the suite until the timeout; without it GUT records the failure and the gate’s own checks fail the run. Tests that need a filesystem path useTestPaths.user_path(...), neverProjectSettings.globalize_path("user://..."): a bareuser://path bypasses the seam and on Windows writes the developer’s real profile. Do not add a second rule -BasePersistenceManager.resolve_store_pathis the store half,history_db_paththe gdext half, andTestPaths.user_dir()asks the sameuser_root_override()for the directory. - The live pane-API smoke is
scripts/smoke_pane_api.py- one harness for every platform, because the transport is the only difference between them (Unix socket vs named pipe; the event client is the same file overopen()).ci-checkand the Linux CI job run it withpython3; thewindows-smokejob onwindows-latestruns the same file withpython, which is the only place the Windows named-pipe control/event listeners and a real Windows pane (default shell%COMSPEC%) are exercised. GUT runs on both: the suite starts panes withSettingsManager.default_shell_command()and asksShellFixtures.print_text()for a command line that emits exact bytes on either platform (POSIXprintf '%b', Windows PowerShell[Console]::Write), so thewindows-smokejob runsscripts/gut-checktoo (under Git Bash - the gate is one bash script for both platforms). Windows gaps pend rather than fail, and each pend names its gap (a fake adapter written as a POSIX shell script; a world-writable file mode, which is not a Windows notion; the threegpty_statedeclaration tests, where ConPTY consumes the OSC):gut-checkprints every pending test and fails on[Risky](a test that ran without asserting), so an explicitly-pended platform gap cannot hide a suite that stopped asserting. Pending is a Windows-only allowance: outside Windowsgut-checkfails on any[Pending]test, because there it can only mean a skip nobody decided on (the extension is a build requirement, so a test that needsGptyAiorGptyCliViewasserts it is present instead of standing down). A step that waits for a program the pane has to launch sizes its budget for that launch, not for the round trip the step is about:nodeoutran the 10 s the extension step used to allow onwindows-smoke(the pane tail held the echoed command line and nothing else,idle_ms~9.8 s), so the step reported “the event never arrived” for a Node.js that was still starting - the same lesson as the GUT fixture’s coldpowershell.exe. Thecli_viewstep runs in the same harness: it creates a pane throughnew-pane --type cli_view --command <program> --arg …and asserts the child’s stdout arrives inpane-read, which is what proves the argv spawn surface live on whichever platform runs the smoke. The concept step seeds one concept into the run’s sandboxed user root before launch - the only registration route that exists, sinceconcept toggleflips a shipped rule rather than adding one; the concept triggers on a marker the shell builds at runtime (the trigger demands the appended digit, so no typed line can match it), and the step asserts the routed{type: concept, name, target}event on the event socket. That is the Windows proof for the ConPTY parser fix: the event can only come from a committed output line, which is exactly what ConPTY’s cursor-move line breaks used to suppress. The final step seeds a plugin record into the sandboxed state store and assertslayout listoffers the plugin’s profile andlayout loadrestores its tile, which is the live proof of the profiles migration’s read path: store →installed_plugin_profiles()→ProfileManager→layoutList/layoutLoad. Every one of those steps is also a CLI teardown sample, and the pane-status stage adds one deliberately: 100 throwawaypane-statusprocesses (≈ 200 connections - each process opens the best-effortversionprobe and then its command), the amplifier for the one WindowsSTATUS_HEAP_CORRUPTION(0xC0000374) the smoke saw on aninject; a nonzero exit fails there with its hex status, and it is never retried. - Media backend tests (
crates/gpty-media/tests): committed fixtures generated from FFmpeg’slavfisources cover every allowed container/codec (H.264/HEVC/VP9/AV1/Theora video; Vorbis/Opus/FLAC/MP3/AAC/PCM audio). They assert probe metadata, packed RGBA/PCM output at the requested rate/channels, seek landing at or after the target, the path gate, clean refusal of garbage/truncated files, close-while-queued (no deadlock), and the two seam checks -build_recipe.rs(components.txt ↔ runtime allowlists) andbinding_rules.rs(banned FFmpeg APIs). They link the pinned FFmpeg: run./scripts/media-depsfirst (every CI job that compiles or loads the extension does). - Seam contract tests: a contract written in one module and implemented in another needs a test that reads both sides, never a copy of the other side written into the test - the copy passes while the shipped code drifts.
crates/gpty-cli/src/commands/mcp.rsparses the real IPC registrations (ipc.rs), the real dispatch (ipc_handlers.gdarms plus themethod ==requestsworkspace.gdintercepts) and everyclient.callliteral undersrc/commands/, and asserts they agree with the advertised MCP tool names;godot/tests/integration/test_ipc_dispatch_contract.gddrivesWorkspaceIpcHandlers.handleagainst a realWorkspace(one instance for the script: each schedules a deferred concept push that must land while it is alive) instead of reimplementing the handlers, andscripts/smoke_pane_api.pyprovespane-read’s scrollback claim live, where output timing is real.