onboard/tests

Covers the onboarding wizard: onboard/config.py (field and section specs), steps.py, actions.py, checks.py, executor.py, prompt.py, plain.py, api_keys.py. Also covers stack.py and the root upgrade.py, which reuse those actions for upgrades and service control.

python -m pytest onboard/tests

onboard/ has an __init__.py, so this suite does not collide with mcp/tests and can be collected alongside it. conftest.py holds the shared fixtures.

---

Structure and flow

File	Guards
test_steps.py	Step-order invariants — build and container-start steps must come last; directories and database must precede them, because Docker creates missing bind-mount targets as root The auth step sits after database (the password lives in it) and before every container step, because Compose refuses to start a service whose token file is missing.
test_fields_for_step.py	Every section maps to exactly one step (SECTION_TO_STEP), and fields_for_step drives the actual prompting. This is the single chokepoint where advanced and condition filtering happens. It also pins the required set — the ElevenLabs key and the OpenAI key — and that the voice roster and default voice are advanced, with defaults that pass the roster parser and put the default voice on the roster, since a skipped question writes its default unchecked. It also pins that the Gemini key stays optional, its help naming mining and review and nothing about images. OpenAI is the only image provider, so its help must say it draws every scene image and that model access is per project, the one failure onboarding cannot check for free; the old "optional — leave it blank" advice would now produce an install that cannot draw a single scene
test_executor.py	Executor is the only dry-run chokepoint: in dry mode it records intents and performs zero real mutations; in live mode it actually acts
test_config.py	Default loading and pre-fill — from a seeded settings table, falling back to each field's declared default. A new install with no database is offered the pre-loaded voices, and a stored empty roster stays empty, so voices someone deleted never come back on a re-run
test_state_migration.py	Bringing an older .onboard-state.json forward as steps are renamed or split
test_bootstrap.py	onboard/bootstrap.py, the venv and dependency bootstrap that runs before anything can be imported — so its two failure messages are the whole interface for a Python that cannot host the wizard. Debian and Ubuntu ship venv with the base interpreter but package pip's bootstrap (ensurepip) separately, which makes both failures routine on a fresh server: python3 -m venv creates the directory, fails at the pip step and leaves an interpreter with no pip beside it. Guards that the hint names the apt package when ensurepip is what is missing and never repeats the python3 -m venv command that just failed, that it stays generic for any other failure (nobody should be sent to apt for an unrelated problem), that a virtualenv holding an interpreter but no pip is created again rather than re-executed into — keying readiness on the interpreter alone reported the same cause later as a different problem, No module named pip — that a venv run reporting success while seeding no pip is still a failure, that a complete virtualenv is re-used without rebuilding it and the caller's own flags survive the re-exec, and that the dependency install failure names the package too. Nothing here runs a real venv or pip: both are patched, and the assertions are on what the user is told and which argv was attempted
test_onboard_cli.py	onboard.py's own argv handling for --install-agent, --uninstall-agent and --list-agents. The module is loaded by path with _ensure_venv and _ensure_deps stubbed, because a real main() would create a .venv in the checkout and pip-install into it before parsing a single flag. An unknown or missing agent name is refused with the supported list; installing and uninstalling both still need a terminal (one asks questions, the other asks for confirmation), while --list-agents is a pipeable report that runs nothing else; and the shared flags reach the install with their real meaning (--disable-verification is verify_keys=False). Also pins that onboarding's POLL_LOG_DIR equals scripts/config.py's, so the cron line's cron.log and the poller's poll.log land in one directory — two hand-kept copies with nothing else holding them together --set-password needs a terminal (it asks twice, hidden) while --rotate-tokens does not, so a script can run it; both dispatch with the shared flags.

Fields and validators

File	Guards
test_agent_model.py	The agent model passed to claude -p … --model <value>. Deliberately permissive — blank means no --model flag at all, and only shapes that would corrupt the argv are rejected
test_agent_install.py	The whole --install-agent flow. Its schema test is the interesting one: it compares AGENTS_SCHEMA against a fully migrated database rather than any migration's source text, because a later ALTER TABLE … ADD COLUMN leaves 0001 untouched — and the comparison includes each column's position, since ADD COLUMN always appends. Also the effort question: skipped by the minimal questionnaire (it carries advanced, which this flow has to honour itself because it builds its own field list), still storing high; asked under --full-questionnaire; and the install-time --effort probe, which runs even when the question was skipped — the default install is the one that never asks and is also the common one — but only warns, never blocks. A test also pins that what a skipped install stores is the field's own default rather than a second copy of the literal. Since the flow became agent-driven it also pins the deployment asymmetries: that a Codex install writes to ~/.codex/skills/, registers MCP with Codex's own argv, and creates no .claude path at all — no commands, no allowlist, no trust record, no second skills copy in the workspace — while every pre-existing Claude assertion stands unmodified, which is what proves the refactor did not change the agent that already worked. Also that the scope question is asked only when the agent has more than one scope, that a stored scope the agent cannot honour is clamped rather than passed to deploy, and that the install-time effort probe is skipped for an agent that cannot reject the flag, since it would be a billed run that could never produce a verdict. A last group covers the post-install checklist, which is adapter-driven for the same reason: a correct Codex install used to report eight phantom problems and tell the user to re-run the install forever, while the skills check itself was a false pass — it tested only that a skills directory existed, and the Remotion installer creates one, so an agent whose own skills never landed still reported installed. It now looks for a named skill in the agent's own root. A last group covers OpenCode, which is the first agent to need a permission document of its own and the first with no MCP-deregistration command: that its install writes to ~/.config/opencode/ and registers with --global at user scope but neither at project scope, that it creates no .claude path, and that the managed rules and the MCP options go into Tubematic's own document (<workspace>/.opencode/opencode.json) rather than any file the operator owns — measured, the workspace's direct file is overridden by a developer's ~/.opencode/opencode.json, while the managed one outranks every other document. Also that the document carries the complete MCP server entry rather than just its options (an options-only entry connects but leaves Code Mode on); that the managed rules are re-asserted at the end, since permissions resolve last-match-wins and a contradicting later rule would otherwise stay in force while a membership test passed; and that the checklist asserts that same shape, and flags a server registered without its Code Mode option separately from one not registered at all, because those two answer different questions. The sharpest one guards ownership: .opencode/opencode.json is a documented OpenCode project-config location and the agent workspace is a directory the operator chooses, and the workspace is Tubematic's own directory, so the document there is written on install and removed on uninstall with no ownership test. The one case worth refusing — a workspace that is already somebody's OpenCode project — is pinned at the workspace question, beside the existing rule that each agent needs a directory of its own, so it runs before anything is copied and needs no gate of its own. Several tests pin what that rule got wrong on the way: the exemption must be Tubematic's own files, since exempting nothing breaks every re-install and upgrade.py with it, while exempting the whole directory lets a document dropped in afterwards survive a redeploy; only a live agents row makes a workspace ours, because an uninstall archives the row and keeps the workspace, so a document there afterwards is the operator's; a foreign document must not relocate the agent, because the pre-fill feeds upgrade.py, which asks nothing; Tubematic writes every basename OpenCode loads, with identical content, since whichever loads last wins — and the checklist watches for a missing one, which reopens that slot; and the rule must cover every basename OpenCode loads, since .opencode/opencode.jsonc is read after .opencode/opencode.json and one rule in the sibling outranks every rule in ours — with a checklist item for a sibling that appears later, which the permissions check alone calls healthy. Uninstalling preserves the operator's comments while still removing the entry, and a document the editor cannot read safely is reported rather than mangled. One more guards the generalised config-root check: it used to compare against a hardcoded ~/.<name>, which is right for Codex and would have reported every ordinary OpenCode install as running under an override MCP registration carries authentication: mcp_auth() names the token file and a helper built from this install's own absolute interpreter and script (quoted, so a path with spaces survives), never the token; Claude's old entry is removed before the new one is added, because its mcp add refuses an existing name; and Codex's helper is patched into config.toml after mcp add — parsed before it is written, refused with a warning naming the 401 when the table is missing, and never written under --dry-run. An OpenCode (re)install runs opencode reload after registering — its background service otherwise keeps a stale, token-less connection — and Claude and Codex, which cache nothing, are never reloaded. Retired skills: _copy_skills only ever adds, so a skill this repository stopped shipping would survive every upgrade and keep teaching the agent a tool the server no longer has. A redeploy at every scope and for every agent removes the names in RETIRED_SKILLS from each skills directory it writes to, and leaves a skill that is not Tubematic's beside it untouched; under --dry-run the removal is only recorded. No retired name may still ship, deployed_paths includes the retired names so an install never upgraded since is still uninstalled cleanly, and uninstall.py's standalone copy of the list must match. The data directory (the database with every API key, and the service tokens) is denied to the agent's reads: Claude's allowlist carries Read(//<abs>/**) — a test pins the double slash, because Claude Code reads a single leading slash as relative to the settings file's project and the rule would deny nothing — merged beside an operator's own denies; OpenCode's managed rules end with an external_directory and a read deny on it (last, since read * is allowed earlier and the last match wins); and both checklists fail when the deny is missing. The same tests pin that the poll logs and the workspace stay outside the denied tree.
test_ai_review_settings.py	The review model, sampling-rate and loop-cap fields, their validators, and the rows apply_pipeline_settings seeds from them. AI review is off by default, and the default lives in four places: the field (whose choices lead with false, because a select falls back to its first choice), both fallbacks in actions.py and SETTINGS_SPEC. A skipped or garbled answer must land on off — the apply-side fallbacks are the easy miss, since drifting them back to true would switch review on for exactly the installs that never answered
test_render_timeout.py	The minutes↔seconds conversion at the load and write boundaries, plus the one-minute floor
test_retention_days.py	The retention_days field and the row apply_pipeline_settings seeds from it. The field is advanced, so a minimal install never asks it and the seed comes entirely from the apply step's fallback literal — a second copy of the default, which must equal SETTINGS_SPEC's or a skipped question silently seeds a different number. 0 is seeded as 0, never clamped, because it is the off switch and a floor of 1 would quietly turn deletion back on. The bounds and the validator mirror the settings API's, so onboarding cannot accept a value the dashboard would refuse
test_voices.py	The ElevenLabs voice roster and default-voice dependent pair, corrupt stored-value handling, and the default-voice fallback. Every voice carries its own model, and onboarding writes no install-wide model row. The trap it guards: onboarding reads stored values before its database step migrates, so on a re-run against an older install the roster is still {name, voice_id}. The strict parser would read that as empty and the apply would wipe the real roster, so load_current_values stamps each legacy voice with the install's old model first, falling back to the default for a blank or retired one exactly as migration 0006 does
test_image_generation.py	The image model, a plain select over the OpenAI catalog with no provider question, and the OpenAI-key gate in apply_install_config: a missing key fails the step even with a Gemini key, after everything typed has been stored for the re-prompt. No image_provider row is seeded. A stored retired model (gpt-image-2, a Gemini model) is corrected to gpt-image-2.5-flare on a re-run: onboarding reads settings before its database step migrates, so it can still see one
test_api_keys.py	Live key validation. Every "invalid" case replays the body a provider actually returned to a deliberately bogus token — these are observed, not read off a docs page. Gemini is probed only for the text model mining and review use: its image probe went with the Gemini image models, and a free-tier key must not be warned about a capability nothing asks it for

What each action writes

File	Guards
test_install_config.py	Configuration is seeded into the settings table, not written to .env. The step refuses to finish without an OpenAI key, Gemini or not, since no scene image can be drawn without one
test_database_step.py	apply_database defers entirely to tubematic_core.db.migrate() rather than asserting a schema of its own — it used to be a fourth hand-maintained copy, already stale by six columns and three tables. Also guards that a dry run creates no database file
test_apply_cron.py	A single poll.py cron line, with no separate research entry. subprocess.run for the read-only crontab -l is monkeypatched, so the suite never touches a real crontab
test_volume_reconcile.py	Named volumes must be recreated when their bind device moves. Docker fixes a volume's driver_opts at creation, and Compose answers "no" non-interactively when asked to recreate
test_renderer_step.py	The renderer step, actions.py::apply_renderer: one image build covers all three services, and tubematic-api, tubematic-mcp and tubematic-renderer are all started. It waits on two healthchecks, because since the MCP server got its own container a healthy Flask no longer says the agent's tools are reachable. _await_health names the container that is actually unhealthy — a combined check that pointed at the API for an MCP failure was useless — and a container still starting at the timeout is a warning, not a failure, since it may yet come up. A sick API ends the step before the MCP wait, rather than spending another minute on a container gated behind it. A dry run stops before any waiting
test_compose_topology.py	docker-compose.yml's topology, parsed as YAML rather than through docker compose config (which needs a daemon and interpolates ${HOME}). Four services, one process each: Flask and MCP once shared a container, and when Flask died MCP still held PID 1, so the container stayed up and restart: unless-stopped never fired — a dead dashboard behind a live-looking container. Pins that the API and MCP run through the shared entrypoint.sh while the renderer bypasses it, that MCP reuses the API image, and that each healthcheck probes only its own port with curl -fs (asserting the port, not the loopback spelling, which had to change). The dependency direction: Flask depends on nothing, so the dashboard and poller work with MCP down; MCP waits for a healthy API at startup only. Ports bind to loopback, because MCP spends money and renders agent-authored scene code with no authentication, and the renderer publishes none. MCP's mounts are an exact set, so adding one forces a yes-or-no answer; every service that logs mounts the shared logs directory, while the dashboard mounts neither logs nor backups (it reads logs over HTTP and must have no route to a copy of the credential store). No service reintroduces env_file, and the Dockerfile CMD names a process, since entrypoint.sh ends in exec "$@"
test_data_dir_checks.py	The prerequisite check's write probe on each ~/.tubematic-data subdirectory. An existing directory this user cannot write or search is a warning with a sudo chown hint, never an error, since the containers run as root and keep working. Write without search (0600) is covered on purpose: it cannot create a file, yet a write-only probe passed it. A missing one is not judged, because the directories step creates it with the right owner. The probe and the directories step share one list (DATA_SUBDIRS), so they cannot disagree. The names are patched on onboard.checks, where they are looked up; patching onboard.config would silently probe the real home directory
test_auth_onboarding.py	The onboarding side of authentication. The auth step creates the tokens and asks for the password only when none is set, so a re-run never silently changes it; under --dry-run it asks nothing and writes nothing; and the database holds the hash, never the password. The Executor's two secret operations record the directory and the token names, never a value. --set-password works before an upgrade (a 1.2.x install has no secrets directory yet), and changing an existing password rotates session_secret, which signs every dashboard session out. --rotate-tokens waits for in-flight work and changes nothing when there is some, rewrites every token in place (same inode), restarts the three services that read one, and reloads only the agents that cache it (OpenCode), never an archived one. The checklist reports a missing password or token, a world-readable secrets directory (as a warning), runs the real header helper, and probes MCP live: a stopped stack is unknown, never open A rotation whose restart or OpenCode reload fails exits non-zero: the files already hold the new tokens, so a zero exit would report a half-rotated, unauthenticating install as a success.
test_stack.py	The stack control upgrade.py and scripts/service.py share. The in-flight gate: a video RENDERING or VOICING, a publish whose upload has not finished (a publication row with no post id yet — one whose post id is set is only polled, which survives a restart — or a PUBLISHING video with no rows at all, which is how an upload in progress looks on a 1.0.0 install, the code an upgrade to 1.0.1 meets), or a held poll lock each block, and so does an agent session a dead poller left running (read from the poller's session registry, a reused pid never mistaken for one), and --force proceeds while still naming the work. An unreadable database also blocks, with a plain "cannot read the database" message and never a raw SQLite error, because WAL mode needs write access to its sidecar files even to read. The lock and the session registry are pinned to the paths scripts/poll.py uses — any other path would exclude nothing — and the lock file must outlive every holder, for the reason poll.py never unlinks it. start must create the data directories and reconcile the volumes before up: these are local-driver bind volumes, which fail the mount rather than create a missing path, and Compose will not move a volume whose device changed. stop must issue docker compose stop, never down. Reconciling a moved volume runs docker compose down, which takes down every container whatever service was named, so when it would and something is running, start must get a yes from its caller's gate first (a required argument, so no caller can forget it), and afterwards brings back every service, not only the named one. A requested service that fails its health check, or a renderer that is not running, fails the start rather than warning, because callers report success from the exit code. The status report must survive a machine with no database, no crontab and no Docker, since it is what someone runs when unsure whether an upgrade worked. A database whose schema is newer than this code's migrations is reported as incompatible, never healthy: older code may not understand a renamed or dropped column Starting the stack creates any missing service token just before up — and only after the stale-volume question, so a start that is refused changes nothing.
test_upgrade.py	upgrade.py, loaded by path. Every step records itself on one dry-run executor, so a single list proves the order: images, then stop, then directories, volumes and up, then cron. Stopping before the start is what keeps new code from migrating the database under old code. The gate refuses before anything changes, and is checked again after the images, because a long build is exactly when a queued render can start — the late-render test inserts one mid-build and asserts nothing is stopped. tubematic_core.db.migrate is replaced by a stub that fails the test: the containers apply migrations under their own lock, and a fourth process racing them adds a failure mode. Only non-archived agents are redeployed, and a missing agent binary warns without failing the upgrade. The previous install is read from the old cron line before it is rewritten, and its .onboard-state.json is carried forward. Only the steps an upgrade actually re-ran are marked done, never install_config. It refuses outright when the database is newer than this version's migrations — running an older version's upgrade.py there is a rollback, which needs the matching backup restored first. The command runs with no terminal, which is the reason it exists. Each behaviour was checked by breaking the code and watching the test fail Authentication: with no dashboard password the upgrade refuses before changing anything and names --set-password (it never asks a question itself); --no-stop is refused while a token file is missing, because old and new containers would refuse each other's calls; missing tokens are created before up; and an agent left without a working MCP registration — a missing binary, or a redeploy whose registration carries no token — still lets the rest of the upgrade finish but fails it, naming the agent, since the server would refuse every call it makes. --check fails on an auth problem alone. OpenAI-only images: with no OpenAI key stored — absent, blank or whitespace — the upgrade refuses before changing anything and points at the Settings dashboard, and a Gemini key alone does not satisfy it; otherwise it would succeed and then fail every scene image on a video whose voicing was already paid for. --no-stop is refused while migration 0016 is pending, because old containers recreated one at a time would keep reading the image_provider row it deletes; a normal upgrade across it is not, and the constant naming the migration is checked against the real file. Removed GifFX component: the renderer compiles every generated scene together, so a scene that still imports installed/GifFX would fail every render on the install — the upgrade refuses before changing anything and names the file, and passes with clean scenes or none at all. The check is a small lexer over the scene, not a substring search: module paths are read only in code, so comments and strings never count. Normalised spellings (installed/./GifFX, installed//GifFX, GifFX.tsx, a/../), a multi-line import, export … from, a bare import, a dynamic import() and require() all refuse, as do the other call shapes webpack bundles ((require)(…), require?.(…), require.resolve(…), require.resolveWeak(…), and the bracket spellings require["resolve"](…), require["resolveWeak"](…) and (require)["resolve"](…), comments around the key included) and — deliberate fail-closed choices, since the lexer cannot tell them from webpack's — loader.require(…) and a call on any other key of require (require["cache"](…), require[key](…)) whose argument names GifFX; and so do a block or line comment between the keyword (or () and the string, a webpack magic comment holding its own quoted string, import type, and an import following a "https://…" string on the same line (the guard against ever "simplifying" to a string-unaware comment strip, which would swallow it). Escapes in the path (\u0047, \u{47}, \x47, an identity escape, a line continuation, inside a no-substitution template too) are decoded before the comparison. A computed import()/require() argument — "../../installed/" + "GifFX", a ${"GifFX"} template, a variable plus "GifFX" — refuses when its string parts, joined, spell GifFX, because webpack folds constant parts into one path. A computed path naming only a sibling ./GifFX refuses too, because a computed path is not resolved: a known limitation, pinned so that changing it is deliberate. A path with an escape that cannot be decoded (\8, a short \x, \u{110000}), in a plain or a computed argument, refuses with a message saying so — fail closed, since it could be hiding the path. Negative controls pass: GifFXLegacy, GifFXNew, ImageFX, a local ./GifFX, a comment naming GifFX, a string holding //, an ordinary \u2014 escape, a computed import() of ImageFX, a bracket key naming GifFX (delete require["cache"]["…/GifFX"]), require["cache"]("./local"), registry["resolve"](…GifFX…) on another object, and the whole old import kept as a note — in a line comment, a one-line or multi-line block comment, a quoted string, a template, or a plain string holding just the path — since none of them imports anything. The lexer is heuristic about JSX and regexes, so those tests pin the direction a misreading may go: an apostrophe, backtick and src/* in JSX text open nothing, so a real require after them still refuses while the same scene with only a commented-out import passes; a regex holding a quote or a backtick is not a string or a template; a stray quote (text the lexer failed to see as JSX) never hides the lines after it. A generic arrow's type parameters (<T extends unknown>, <T,>, <T extends object = {}>, <const T extends …>, and <É extends unknown>, since a name may start with any Unicode letter, as a tag may) keep the lexer in code: an escaped, joined or template-built import after one refuses, a note in a comment or string after one passes, and a require(pick({…})) after one passes — read as JSX text, that call would be cut off at its { and refuse, which is what happens after a bare <T> (JSX in TSX), the stand-in for code misread as JSX. Text misread as JSX is lexed as code, so the same imports after a bare <T> refuse and the same notes pass; JSX text holding only import words passes, while a call a {…} cuts off there refuses because it cannot be judged. Misread text is lexed in pieces split at every < and {, even one inside a comment or a bracket key, so a piece that ends — or ends in a comment — while import or require awaits its path or call refuses: require /* < */ (…), import /* { */ (…), require // <, require /* < */ .resolve(…), require["<"](…) import G from /* < */ "…" and import G from // < all refuse after both the Unicode generic and the bare <T>. Its known cost is pinned: JSX text ending in require. before a tag refuses, while <p>Packets from {src} to {dst}</p> passes, since from refuses there only before a comment. A comment, template, attribute value or closing tag that never closes hides nothing: escaped and joined imports after one refuse, notes after one pass. Every .tsx in the repository scans clean. Removing the comment skipping, the string-versus-module-path distinction, the computed-argument check, its undecodable refusal, the JSX tracking, the regex handling, the stray-quote recovery, the generic-arrow recognition, each call-shape rule, the bracket-key rule, the Unicode generic name, lexing misread JSX text as code, the cut-off refusal, each part of the refusal of a piece that ends awaiting a path (an open bracket key, a pending word, a comment cut off after one — line or block), or lexing on past an unclosed comment, template, attribute value or closing tag was each checked to fail these tests. An unreadable scene refuses too (fail closed: the renderer could not compile it either). A scene planted mid-build — the old MCP server keeps serving — is refused by the second check, after the last work gate and before the stop, with nothing stopped, with and without --no-stop; removing that second check, or the unreadable refusal, was checked to fail these tests. A scene-config.json that names GifFX only loses that overlay if its video is rendered again (the renderer skips a scene it cannot load), so it is a warning and the upgrade continues.
test_install_mode.py	Dev checkout vs. release tarball. Mode is auto-detected from renderer/Dockerfile's presence (a release tarball ships no source trees), overridable with --install-mode; a release install pulls container images instead of building them. It used to skip the Node/npx prerequisite checks too, and a test pinned that; both now run in every mode, because the host-side consumer of Node is the Remotion skills installer that --install-agent runs whichever way Tubematic was installed, not the local build — so skipping them hid a prerequisite a release install needs just as much, and the user met it later as a warning from a different command. Also guards that dev-mode behavior is untouched, that the resolved mode is both printed and recorded in .onboard-state.json, that a later flagless run (e.g. --step renderer) reuses the recorded mode instead of re-detecting, that an explicit --install-mode still overrides a recorded one, and that a failed release-mode pull shows Docker's own reason rather than the build-path's docker compose logs hint (which would show an empty log — a pull failure creates no container), and never a login instruction: the images are public. Its prerequisite/installation stubs patch onboard.plain.*, never onboard.checks.*: plain binds both names at import, so patching the defining module leaves those aliases pointing at the real functions and the stub never runs. Each site asserts the mock was actually called, because an inert patch does not fail — it silently exercises the real check and passes wherever Docker happens to be running. The Node check compares major and minor against 22.20, the Remotion skills installer's floor: a check on the major alone passed 22.0, which then crashes that installer. It stays a warning, never blocking onboarding

Presentation

File	Guards
test_prompt.py	The pure logic in prompt.py — real questionary interaction is stubbed, so this covers field dispatch, the per-step summary, and secret masking in that summary. With nothing stored, a blank roster answer is accepted rather than re-asked: on a new install the current value is the pre-loaded pair, so an empty one means voices were removed on purpose. The roster prompt asks each voice's model, offering the model that voice already had, else the previous voice's, so a roster on one model is a string of Enters The full questionnaire's field count for the pipeline-settings step is pinned alongside the minimal list, so a new advanced field (the Auto Pilot question is one) has to be added to it deliberately.
test_auto_pilot.py	The Auto Pilot question. It is advanced, so the default questionnaire never asks it; it is off in the field default and in SETTINGS_SPEC alike; a skipped question seeds it off; and only an explicit true seeds it on — a blank, a yes or a 1 all read as off, because this is the switch that removes every human checkpoint. The question's help must carry the warning
test_plain.py	plain.py regressions, including two real bugs found in review of the questionary rewrite

---

What these encode

The API-key rejection signatures are observations. Each provider answers a bad key differently, and two of the differences are traps the tests pin:

• Gemini answers 400, not 401. A shared "401/403 means invalid" rule silently accepts every bad Gemini key. The discriminator is error.details[].reason == "API_KEY_INVALID", never status.
• OpenAI's 401 message echoes a masked slice of the submitted key — twelve real characters of a live secret. It is the one provider message that must never be surfaced or logged.

A pasted key fails in two ways that do not look like failures. httpx.LocalProtocolError subclasses httpx.HTTPError, so a routine handler catches a trailing newline off a terminal copy and reports it as a network problem. A non-ASCII character raises UnicodeEncodeError, which is not an httpx exception at all and escapes such a handler entirely. Both are covered.

load_current_values() pre-fills every field before any question is asked. That is why a skipped question is identical to answering it with its default, and why no apply-side plumbing was needed for minimal onboarding. The corollary is the trap: every apply_* action's v.get(key, "<literal>") fallback is a second copy of each default, and must move with the field spec or a skipped question silently seeds the old value.

Secrets are masked in the end-of-step summary. That summary prints to scrollback, so a test pins it. Do not "simplify" _display_answer.

The questionary Style lives in prompt.py, not plain.py. --status and --steps import plain.py and are documented as pipeable non-interactive reports, so that module must not start requiring the interactive prompt library.

Prompting is never exercised against a real terminal. onboard.py blocks non-interactive runs, so colours, glyphs and live prompt behaviour are only ever driven with ask_field stubbed. Any claim about how the wizard looks needs a human in a TTY.

Install-mode precedence is explicit flag, then recorded state, then live detection. Without the middle tier, a later flagless run inside a real dev checkout (e.g. onboard.py --step renderer) would always re-detect "dev" — even for an install that was deliberately done in release mode — since detection only ever looks at renderer/Dockerfile, not at what actually happened before. Recording the resolved mode once and reusing it is what lets a later step agree with the run that came before it; an explicit --install-mode still wins over both.

A failed release-mode pull shows Docker's own reason. The hint used to be copy-pasted from the build path (docker compose logs <service>), which shows nothing at all for a pull failure — no container was ever created. When the images were private it pointed at docker login ghcr.io; they are public now, so a failed pull is a network problem or a version with no published images, and the hint (actions.py::_pull_failure_hint, shared with upgrade.py) shows the tail of Docker's error plus where to look. The test also pins that it never tells anyone to log in.