agent-desktop/docs/plans/2026-06-29-001-feat-wait-for-selector-flag-plan.md
Lahfir dde0b98dc0
Some checks are pending
CI / Format (push) Waiting to run
CI / Rust 1.89 MSRV (push) Waiting to run
CI / Native check (macOS) (push) Waiting to run
CI / Native check (Linux) (push) Waiting to run
CI / Native check (Windows) (push) Waiting to run
CI / Test (push) Waiting to run
CI / Test (Linux) (push) Waiting to run
CI / Test (Windows) (push) Waiting to run
CI / FFI Python Smoke (push) Waiting to run
CI / FFI Header Drift (push) Waiting to run
CI / FFI Panic Guard (push) Waiting to run
CI / FFI Stub-Adapter Passthrough (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
CodeQL / Analyze (rust) (push) Waiting to run
Release / Release Please (push) Waiting to run
Release / Build (aarch64-apple-darwin) (push) Blocked by required conditions
Release / Build (x86_64-apple-darwin) (push) Blocked by required conditions
Release / Build FFI (aarch64-apple-darwin) (push) Blocked by required conditions
Release / Build FFI (x86_64-apple-darwin) (push) Blocked by required conditions
Release / Build FFI (x86_64-unknown-linux-gnu) (push) Blocked by required conditions
Release / Build FFI (aarch64-unknown-linux-gnu) (push) Blocked by required conditions
Release / Build FFI (x86_64-pc-windows-msvc) (push) Blocked by required conditions
Release / FFI Release Gates (push) Blocked by required conditions
Release / Publish to GitHub Release (push) Blocked by required conditions
Release / Publish to npm (push) Blocked by required conditions
Release / Publish Skills to ClawHub (push) Blocked by required conditions
Supply Chain / Audit (push) Waiting to run
chore: track planning artifacts with secret and privacy scanning (#109)
Tracks docs/plans and docs/brainstorms, adds .gitleaks.toml with privacy rules, and runs a history-mode gitleaks scan in CI. Docs and tooling only - no releasable change.
2026-07-26 01:41:55 -07:00

32 KiB
Raw Blame History

artifact_contract artifact_readiness execution type product_contract_source origin created
ce-unified-plan/v1 implementation-ready code feat ce-plan-bootstrap https://github.com/lahfir/agent-desktop/issues/84 2026-06-29

feat: add --wait-for selector polling flags

Summary

Add global --wait-for / --wait-for-gone (with -w short for --wait-for) and --wait-timeout flags that poll the accessibility tree until an element matching a compact selector appears (or disappears), then return the resulting snapshot. This collapses the agent-side "snapshot → check → sleep → retry" loop into a single invocation, and lets action commands confirm a post-action UI change in one call (click @e5 -w ":Saved!").

The feature is pure CLI orchestration over existing engines — it reuses find's matcher and wait's poll loop. No new platform/AX code, no CSS parser, no new adapter methods.


Problem Frame

Today an agent that needs to wait for a UI state change (modal appears, spinner disappears, button becomes available) must hand-roll a polling loop: snapshot, inspect JSON, sleep, retry. That burns tokens, adds latency, and is re-implemented in every agent workflow.

The engine to avoid this already shipped in 0.4.4 as the wait command (wait --text/--window/--element … --timeout), which polls with 200ms→1s backoff and returns a kind: "wait_timeout" error envelope on expiry. What's missing is ergonomics: a single-string selector and a flag that fuses the wait onto the command that produces the snapshot, so click; wait; snapshot becomes click … -w "…". Issue #84 asks for exactly this, and explicitly frames it as small CLI-only work.

The issue's button:contains('Submit') spelling is illustrative, not a contract — its own "Key design points" say "Reuses existing selector/filter infrastructure," which is find's --role/--name/--text/--value + node_matches. We honor the latter.


Requirements

  • R1. A global --wait-for <SELECTOR> flag (short -w) blocks until an element matching <SELECTOR> is present in the polled tree, then returns the snapshot.
  • R2. A global --wait-for-gone <SELECTOR> flag blocks until no element matches <SELECTOR>, then returns the snapshot (covers "spinner disappears").
  • R3. A global --wait-timeout <MS> flag bounds the poll (default 30000ms, matching wait). On expiry the command exits 1 with a kind: "wait_timeout", predicate: "selector" error envelope, consistent with wait.
  • R4. <SELECTOR> is a compact role:text string parsed into a FindQuery and matched by the same node_matches used by find. Forms: "button:Submit" (role+text), "button" (role only), ":Saved!" (text only).
  • R5. snapshot --wait-for … polls the snapshot's own scope (--app/--window-id/--surface/depth) and returns the matched snapshot.
  • R6. Ref-resolving action commands (click, double-click, triple-click, right-click, clear, focus, toggle, check, uncheck, expand, collapse, scroll-to, type, set-value, select, scroll) accept --wait-for/--wait-for-gone: perform the action, then poll the app the acted-on ref belongs to (entry.source_app), returning the post-action snapshot with the original action result preserved.
  • R7. --wait-for and --wait-for-gone are mutually exclusive (clap conflicts_with, exit 2); --root and --wait-for/--wait-for-gone are mutually exclusive on snapshot (ref-rooted drill-down has no poll semantics).
  • R8. Passing --wait-for/--wait-for-gone to a command that does not support it (anything other than snapshot and the R6 action commands) is an INVALID_ARGS error, not a silent no-op. The supporting set is a single &[&str] checked at dispatch time.
  • R9. Batch mode does not honor an outer --wait-for; batch items run independently and never inherit the CLI-level selector.
  • R10. No new platform-adapter methods; agent-desktop-core remains free of platform crates (CI cargo tree gate stays green).

Success criteria: an agent replaces a multi-call poll loop with one snapshot -w "<selector>"; a click @e5 -w ":Saved!" returns only after "Saved!" is visible; a timeout returns exit 1 with a discriminable envelope.


Key Technical Decisions

KTD1 — Compact role:text selector, not a DSL. Parse on the first : into role (left) and text (right); each side is trimmed and an empty side becomes None (unconstrained on that axis) — never Some(""). This matters: search_text::contains runs haystack.as_bytes().windows(needle.len()), which panics on a zero-length needle, so "button:", ":", and " : " must collapse the empty side to None rather than threading Some("") into node_matches. The result is a FindQuery fed to node_matches. This keeps the single-string ergonomics the issue shows while adding ~5 lines of parsing instead of a CSS grammar that would become a permanent public contract. Role is normalized via roles::normalize_role_query; text via search_text::normalize (the exact path find uses), so selector semantics equal find semantics by construction. node_matches's text axis already searches name/value/description (search_text::node_contains), so window:Settings matches a node whose role is window and whose text contains "Settings"; matching by window title specifically is out of scope (use wait --window for that).

KTD2 — One matcher, promoted out of find. FindQuery and node_matches are currently private in commands/find.rs. Promote them (plus the role:text parser) into a new commands/query.rs; find imports them. This is the anti-redundancy move: the wait path does not get a parallel matcher — both call sites share one. Behavior of find is unchanged (pure move + re-export).

KTD3 — One poll helper, reusing the wait loop shape. New commands/wait_selector.rs mirrors wait_for_text: snapshot::build (cheap, no persist) in a 200ms→1s backoff loop, run node_matches over the tree, persist the refmap and return the snapshot envelope only on match. Presence vs. absence is one boolean (gone). Retryable poll set is wider than wait_for_text'sTimeout, ElementNotFound, AppNotFound, and WindowNotFound are all swallowed and surfaced as last_error on timeout. This is the correctness fix for the common post-action case where the action closes/navigates the target (click @submit -w ":Saved!" dismisses the dialog): without AppNotFound/WindowNotFound in the retryable set, the poll would propagate a raw WINDOW_NOT_FOUND instead of the promised wait_timeout envelope. The empty/match-everything selector guard lives here (before the loop) so both call sites (snapshot self-apply and post-action) reject it identically.

KTD4 — Global flags carried in CommandContext, applied at two scoped call sites, both keyed to the correct app. The flags are defined once as clap globals (like --headed) and threaded into CommandContext (new WaitSelector config, set via a with_wait_selector(...) builder mirroring with_headed). Two thin application points, both delegating to the single wait_selector helper:

  • snapshot::execute self-applies when a selector is set, scoped to its own --app/opts (correct ordering: wait is the snapshot; no wasted eager build).
  • The ref-action path in helpers.rs applies it after the action, scoped to entry.source_app — the app the resolved ref belongs to — embedding the action result under data.after_action.

Why not a binary post-dispatch hook (rejected). A hook in run_with_adapter only sees the action's JSON result, not the ref's pid/app. In the default headless mode AXPress does not foreground the target app, so the frontmost app is usually the terminal, not the acted-on app — a focused-app poll would read the wrong tree and time out on the default code path. The ref-action path already holds the RefEntry (source_app), so scoping there is both correct and the natural place that covers the ref-resolving commands R6 enumerates. This is layering, not duplication — the loop, matcher, and persistence live solely in the helper.

Wiring reaches all 16 R6 commands via the two helpers.rs entrypoints (verified against the codebase): 11 commands (click, double-click, triple-click, clear, focus, toggle, check, uncheck, expand, collapse, scroll-to) go through execute_ref_action_with_context; 5 (type, set-value, select, scroll, right-click) go through execute_ref_action_result_with_context and then serialize themselves. right-click is the only one that builds a custom JSON body (its menu probe), so its wait must run after the probe (action → menu probe → wait). execute_by_ref (the FFI bridge) also routes through execute_ref_action_with_context but is unaffected: the FFI CommandContext never carries a WaitSelector, so apply_post_action_wait early-returns.

KTD5 — Timeout returns the wait error envelope plus the last snapshot's id. The issue says "return the last snapshot on timeout," but the established wait convention is a TIMEOUT error (kind: "wait_timeout") carrying a compact last_error, and finish() maps Err → exit 1. We keep the error-envelope convention (consistency, no bespoke success-on-timeout path) but honor the issue's debuggability intent: on timeout, persist the last successfully-built tree and include its snapshot_id in details alongside last_error, so the agent can inspect the final tree state without racing a fresh snapshot. For post-action timeouts the original action result is also embedded under details.after_action so the caller learns the action ran.

KTD6 — CLI-only; FFI parity deferred. The flags live in the binary/CLI + core helper. The FFI cdylib does not gain a --wait-for surface in this PR (the issue scopes this as "purely a CLI orchestration feature"). The wait_selector core helper is reusable by a future ad_wait_for entrypoint; noted under Deferred.


High-Level Technical Design

                    --wait-for / --wait-for-gone / --wait-timeout  (clap globals)
                                         |
                                CommandContext (WaitSelector)
                                         |
                 +-----------------------+------------------------+
                 |                                                |
        snapshot::execute                          helpers.rs ref-action path
        (selector set?)                            (after action, selector set?)
                 |                                                |
                 | scope = --app/opts                            | action runs first -> ActionResult
                 |                                                | scope = entry.source_app  (correct in headless)
                 +----------------------+   +---------------------+
                                        |   |
                                  commands/wait_selector::execute
                                        |
                  loop: snapshot::build -> node_matches (present|absent)
                        match -> persist refmap, return snapshot envelope
                        timeout -> wait_timeout::selector(...) (exit 1)
                                        |
                                 commands/query.rs
                          FindQuery + node_matches + parse_selector
                                   (shared with find.rs)

Implementation Units

U1. Extract shared selector matcher into query.rs

Goal: Single matcher shared by find and the wait path; add the role:text parser. Requirements: R4, KTD1, KTD2. Dependencies: none. Files:

  • crates/core/src/commands/query.rs (new) — move FindQuery, node_matches, count_matches's predicate use; add pub fn parse_selector(&str) -> FindQuery.
  • crates/core/src/commands/find.rs — delete the moved items, import from query.
  • crates/core/src/commands/mod.rs — register query.
  • crates/core/src/commands/query_tests.rs (new). Approach: Pure refactor + addition. FindQuery::from_args stays in find (it depends on FindArgs); the reusable pieces are FindQuery the struct, node_matches, and the new parse_selector. parse_selector splits once on :; trims each side; an empty-after-trim side becomes None, never Some("") (a Some("") text would reach search_text::contains and panic on windows(0) — see KTD1); role through roles::normalize_role_query, text through search_text::normalize. Also add pub fn tree_has_match(&AccessibilityNode, &FindQuery) -> bool here (short-circuiting presence check) so wait_selector shares the exact predicate. find's public behavior must not change. Patterns to follow: existing FindQuery/node_matches in find.rs; roles::normalize_role_query and search_text::normalize/contains/node_contains. Test scenarios:
  • parse_selector("button:Submit") → role normalized to button, text normalized to submit.
  • parse_selector("button") → role set, text None.
  • parse_selector(":Saved!") → role None, text set.
  • parse_selector(""), parse_selector(":"), parse_selector(" : ")both None (match-everything; rejected by wait_selector, see U2).
  • parse_selector("button:") → role button, text None (not Some("")).
  • parse_selector(" button : Submit ") → trimmed to role button, text submit.
  • parse_selector("textfield:a:b") → splits on first colon only; text is a:b.
  • A FindQuery { text: None } never causes search_text::contains to be called with an empty needle (guards the windows(0) panic).
  • node_matches/tree_has_match over a fixture tree returns identical results before/after the move (regression). Verification: find tests pass unchanged; new parser tests pass; cargo tree -p agent-desktop-core shows no platform crates.

U2. wait_selector poll helper

Goal: The reusable poll-until-present/absent engine returning a snapshot envelope. Requirements: R1, R2, R3, R5, KTD3, KTD5. Dependencies: U1. Files:

  • crates/core/src/commands/wait_selector.rs (new).
  • crates/core/src/commands/wait_timeout.rs — add pub(crate) fn selector(selector: &str, gone: bool, timeout_ms, last_error, last_snapshot_id: Option<String>) -> Result<Value, AppError> (predicate: "selector", kind: "wait_timeout", includes snapshot_id in details when a tree was built).
  • crates/core/src/commands/mod.rs — register wait_selector.
  • crates/core/src/commands/wait_selector_tests.rs (new). Approach: execute(WaitSelectorInput { query, gone, app, opts, timeout_ms }, adapter, context). First, reject a match-everything query (both axes None) with INVALID_ARGS — this single guard covers both call sites (snapshot self-apply and post-action). Loop mirrors wait_for_text: snapshot::build(adapter, &opts, app, None); present = query::tree_has_match(&tree, &query); matched = if gone { !present } else { present }. On match: RefStore::for_session(...).save_new_snapshot(&result.refmap), return a JSON object with the same field shape snapshot::execute emits (app, window, ref_count, snapshot_id, tree) plus elapsed_ms and matched_selector. snapshot::format_result is private — either promote it to pub(crate) and reuse it, or replicate the shape (the field set is the contract). Swallow the retryable set (Timeout, ElementNotFound, AppNotFound, WindowNotFound) into last_error, and on each successful build remember the built tree's persisted id as last_snapshot_id. On deadline call wait_timeout::selector(... , last_error, last_snapshot_id). Reuse the 200ms→1s doubling backoff verbatim. Patterns to follow: wait::wait_for_text (loop, backoff, retryable-error handling, persist-on-match, last_error); snapshot::format_result (envelope shape — promote to pub(crate) if reused). Test scenarios (MockAdapter; deterministic trees):
  • Match-everything query (FindQuery both None) → Err INVALID_ARGS before any poll.
  • Element present on first poll → returns snapshot envelope with matched_selector, elapsed_ms, a persisted snapshot_id.
  • Element absent then present on Nth poll → returns once present (mock whose tree changes across calls).
  • gone=true, element present then absent → returns when absent.
  • gone=true, element never present → returns immediately (already gone).
  • Never matches within timeout → Err code == TIMEOUT, details kind=="wait_timeout", predicate=="selector", last_error populated, snapshot_id of the last built tree present in details.
  • Each retryable error (ElementNotFound, AppNotFound, WindowNotFound) mid-poll is swallowed, not surfaced, until timeout — covers the "action closed the window" case.
  • Persisted snapshot is loadable by the returned snapshot_id (refs usable by a follow-up action). Verification: all unit tests pass; timeout envelope is JSON-discriminable from a chain_deadline timeout.

U3. Global flags + CommandContext wiring + snapshot self-apply

Goal: Define the flags once, carry them, reject them on unsupported commands, and make snapshot honor them in its own scope. Requirements: R1, R2, R3, R5, R7, R8, R9, KTD4. Dependencies: U2. Files:

  • src/cli/mod.rs — add three global = true args: --wait-for/-w (Option<String>, conflicts_with = "wait_for_gone"), --wait-for-gone (Option<String>), --wait-timeout (u64, default 30000). The --wait-for/--wait-for-gone pair is mutually exclusive via clap conflicts_with (exit 2).
  • crates/core/src/context.rs — add WaitSelector { query_raw: String, gone: bool, timeout_ms: u64 } carried in CommandContext; with_wait_selector(Option<WaitSelector>) builder mirroring with_headed; accessor. Update for_batch_item to set wait_selector: None — the struct literal is exhaustive, so this is a required compile fix and the correct semantics (R9: batch items never inherit the outer selector).
  • src/main.rs — build WaitSelector from the parsed globals and attach via with_wait_selector. Before dispatch, when a selector is set, reject it with INVALID_ARGS unless the command is snapshot or one of the R6 action commands (R8) — a single const WAIT_SUPPORTED: &[&str] checked against cmd.name(). This closes the find -w … silent-no-op footgun.
  • src/cli_args/mod.rs (SnapshotArgs) / src/cli/mod.rs — make --root and --wait-for/--wait-for-gone mutually exclusive (R7); since the wait flags are global, enforce this in snapshot::execute (or main) as an INVALID_ARGS check rather than clap (a global cannot conflicts_with a local arg cleanly).
  • crates/core/src/commands/snapshot.rs — when context carries a wait selector and root_ref is None, delegate to wait_selector::execute with snapshot's opts + app; parse_selector the raw string. If root_ref and a selector are both set → INVALID_ARGS.
  • crates/core/src/commands/snapshot_tests.rs — selector-path tests. Approach: Flags are global so they attach to every subcommand with one definition (no per-command duplication; avoids a clap conflict between a global -w and a local one). snapshot::execute checks the selector branch after the existing root_ref early-return guard so the two never silently co-apply; the explicit INVALID_ARGS makes the combination loud. --wait-timeout default 30000 matches wait. Patterns to follow: existing globals (--headed, --session) in src/cli/mod.rs; CommandContext::with_headed and for_batch_item in crates/core/src/context.rs; cmd.name() dispatch in src/main.rs. Test scenarios:
  • snapshot -w "button:Submit" on a tree containing it → returns matched snapshot (MockAdapter).
  • snapshot --wait-for-gone "progressindicator" when absent → returns immediately.
  • --wait-for + --wait-for-gone together → clap parse error, exit 2 (conflicts_with).
  • find -w "button:Submit"INVALID_ARGS (unsupported command, R8), not a silent find result.
  • snapshot --root @e5 -w ":button"INVALID_ARGS (R7), exercised in snapshot_tests.
  • snapshot --app Foo -w "…" polls scope Foo (assert app passed through).
  • Batch item does not inherit an outer -w (assert for_batch_item context has wait_selector: None).
  • snapshot with no wait flags → identical output to today (regression).
  • clap: -w resolves to --wait-for; --wait-timeout 5000 parses. Verification: cargo test -p agent-desktop (binary contract tests) + core snapshot tests pass; --help shows the three flags as global.

U4. Post-action wait in the ref-action path (helpers.rs)

Goal: Ref-resolving action commands perform their action, then wait — scoped to the acted-on ref's app — returning the post-action snapshot with the action result preserved. Requirements: R6, KTD4, KTD5. Dependencies: U2, U3. Files:

  • crates/core/src/commands/helpers.rs — add pub(crate) fn apply_post_action_wait(result: Value, app: Option<&str>, context: &CommandContext) -> Result<Value, AppError>: returns result unchanged when context carries no selector; otherwise runs wait_selector::execute scoped to app (which owns the match-everything guard, U2) and returns the snapshot envelope with result embedded under after_action. On timeout it surfaces the wait_timeout error with the action result in details.after_action.
  • crates/core/src/commands/helpers.rs — in execute_ref_action_with_context, stop discarding _entry: keep it and pass entry.source_app.as_deref() to apply_post_action_wait before returning. This covers the 11 simple ref actions (click, double-click, triple-click, clear, focus, toggle, check, uncheck, expand, collapse, scroll-to) in one edit.
  • crates/core/src/commands/{type_text,set_value,select,scroll}.rs — each receives (entry, result) from execute_ref_action_result_with_context and ends with Ok(serde_json::to_value(result)?) (they do not build custom JSON). Wrap that return: helpers::apply_post_action_wait(serde_json::to_value(result)?, entry.source_app.as_deref(), context) (4 sites).
  • crates/core/src/commands/right_click.rs — the one command that does build a custom JSON body (its menu probe). Call apply_post_action_wait after the menu-probe JSON is assembled, so ordering is action → menu probe → wait (5th bespoke site).
  • crates/core/src/commands/helpers_ref_action_tests.rs (modify — file already exists) — add post-action merge, scope, timeout, and right-click-ordering tests. Approach: One shared wrapper, six call sites (1 helper edit covering 11 commands + 5 bespoke: type/set-value/select/scroll/right-click) — every site already holds the RefEntry, so scoping by entry.source_app is a field read, not a pid→name lookup. Total coverage = the 16 R6 commands. execute_by_ref (FFI bridge) also routes through execute_ref_action_with_context but is behaviorally unaffected: the FFI CommandContext never carries a WaitSelector, so apply_post_action_wait early-returns. Snapshot is untouched here (it self-applied in U3), so there is no double-wait. Scoping by entry.source_app is the correctness fix over a focused-app hook: in headless mode the acted-on app is generally not frontmost. Patterns to follow: execute_ref_action_with_context/execute_ref_action_result_with_context in helpers.rs; the custom-response assembly in right_click.rs; envelope shape in commands/snapshot::format_result. Test scenarios (MockAdapter):
  • click @e5 -w ":Saved!" where the mock tree gains "Saved!" after the action → envelope contains the snapshot and after_action with the click result.
  • Post-action wait is scoped to entry.source_app, not the focused app (assert the app handed to wait_selector equals the ref's source app even when a different app is "focused" in the mock).
  • Post-action selector never matches → Err code==TIMEOUT, details.kind=="wait_timeout", details.after_action carries the action result, details.snapshot_id present.
  • Action that closes the source app/window mid-wait → times out with the wait_timeout envelope (AppNotFound/WindowNotFound swallowed, U2), not a raw WINDOW_NOT_FOUND.
  • An action command with no wait flag → result returned byte-for-byte unchanged (regression).
  • right-click @e -w ":menuitem" → wait runs after the menu probe; after_action carries the right-click body including its menu field.
  • A simple ref action (toggle), a bespoke serialize-only action (type), and right-click all merge correctly (covers all three wiring paths). Verification: cargo test --lib -p agent-desktop-core ref-action tests pass; cargo tree -p agent-desktop-core clean; real smoke deferred to E2E (U5).

U5. Docs, help text, and E2E coverage

Goal: Document the flag and prove it against a real app. Requirements: R1R6 (user-facing surface). Dependencies: U3, U4. Files:

  • skills/agent-desktop/references/commands-interaction.md and the observation reference — document --wait-for/--wait-for-gone/--wait-timeout, the role:text selector, that post-action waits poll the acted-on ref's app (entry.source_app), the timeout envelope (with snapshot_id of the last tree), the unsupported-command INVALID_ARGS (R8), the --root exclusion (R7), and that batch does not honor an outer -w (R9). Note the canonical pattern for unsupported commands like find: snapshot -w "<sel>" then find.
  • src/cli/help_after.txt — add a --wait-for usage example if examples live there.
  • tests/e2e/run.sh + fixture interactions — one appearance wait and one disappearance wait against the SwiftUI fixture, verified by observation (a control that appears/disappears on action). Approach: Documentation-only + E2E. Keep help text terse; the skills reference carries the detail. E2E asserts by independent observation per the harness contract, not the command's own ok. Patterns to follow: existing --force documentation added in 0.4.4; tests/e2e/README.md conventions. Test scenarios: Test expectation: none for docs. E2E: (a) trigger a control that reveals a "Saved!" label, assert -w ":Saved!" returns only after it appears; (b) trigger a spinner that clears, assert --wait-for-gone returns after it's gone; (c) timeout case returns exit 1 with the wait_timeout envelope. Verification: bash tests/e2e/run.sh green (headless + --headed); skills reference renders.

Scope Boundaries

In scope: the three global flags, the role:text selector, the shared matcher extraction, the poll helper (incl. window-close-tolerant retryable set and timeout snapshot_id), snapshot self-apply, post-action wait for all 16 R6 commands, the unsupported-command INVALID_ARGS guard (R8), the --root exclusion (R7), batch isolation (R9), docs + E2E.

Deferred to Follow-Up Work

  • FFI ad_wait_for entrypoint — the core wait_selector helper is reusable; a cdylib surface is a separate, additive PR (KTD6).
  • Richer selector axes (name:, value:, state: predicates, multiple constraints in one string) — only role:text ships now; the matcher already supports name/value, so a future grammar can expand without re-architecting.
  • --wait-for on non-ref commands (e.g. launch Foo -w "button:Login", find) — rejected with INVALID_ARGS this PR (R8); the workaround is two calls (launch Foo; snapshot --app Foo -w "button:Login"). Extending support to find/launch is additive follow-up.
  • --wait-for in batch mode — batch items run independently and ignore an outer -w (R9). Per-item wait selectors in the batch JSON schema are deferred.
  • Polling-interval flag — backoff is fixed (200ms→1s) to match wait; expose only if measured need arises.
  • --wait-for-gone stabilization guard — see Risks; an optional "require one present observation before accepting absent" guard is deferred unless flakiness is observed.

Out of scope (not this product)

  • CSS / :contains() selector grammar (KTD1).
  • A success-on-timeout return path that diverges from the wait envelope (KTD5).

Risks & Dependencies

  • R: post-action wait reads the wrong app's tree (headless default). Resolved by design: scope is entry.source_app (the resolved ref's app), not the focused app — a binary post-dispatch hook was rejected for exactly this reason (KTD4/U4). Test asserts the ref's app is used even when a different app is focused.
  • R: post-action wait errors instead of timing out when the action closes the window. Resolved: AppNotFound/WindowNotFound are in the retryable poll set (KTD3/U2), so a window-closing click @submit -w ":Saved!" times out with the wait_timeout envelope (action result preserved) rather than a raw WINDOW_NOT_FOUND.
  • R: empty-needle panic in the matcher. Resolved: parse_selector collapses an empty axis to None (KTD1/U1) so search_text::contains is never called with a zero-length needle (windows(0) panic); U1 tests cover "button:", ":", " : ".
  • R: --wait-for-gone false-positive on a transient empty/partial tree (first poll fires before the action's AX notification propagates, reads no match, returns "gone"). Confidence moderate. Mitigation this PR: documented limitation + the standard 200ms first-interval gives the tree time to populate; a "require one present observation before accepting absent" guard is deferred (Scope Boundaries) pending observed flakiness.
  • R: find regression from the matcher move. Mitigation: U1 is a pure move with node_matches/tree_has_match parity tests; find's existing suite must pass unchanged before U2 starts.
  • R: 30s default --wait-timeout is long for a blocking post-action call in an agent pipeline. Mitigation: matches wait for consistency; agents set --wait-timeout explicitly for tight loops. Documented in U5.
  • R: global -w short-flag collision. Verified clear — only -v (global) and -i (snapshot) exist today.
  • Dependency: none external; reuses snapshot::build, RefStore, node_matches/tree_has_match, wait_timeout, and RefEntry.source_app.

Definition of Done

  • All five units landed; cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings, cargo test --lib --workspace, cargo test -p agent-desktop, and cargo tree -p agent-desktop-core (no platform crates) green.
  • snapshot -w "<role:text>", --wait-for-gone, and post-action click @e -w "…" work against the E2E fixture in headless and --headed mode.
  • Timeout exits 1 with a kind:"wait_timeout", predicate:"selector" envelope carrying the last tree's snapshot_id and last_error; post-action timeout preserves the action result in details.after_action.
  • All 16 R6 commands honor --wait-for (verified: 11 via execute_ref_action_with_context, 5 bespoke incl. right-click); find/launch/other unsupported commands return INVALID_ARGS (R8); batch ignores an outer -w (R9).
  • --wait-for + --wait-for-gone together → clap parse error (exit 2); --root + --wait-forINVALID_ARGS (R7); a match-everything selector → INVALID_ARGS.
  • Skills reference and --help document the flags; issue #84 referenced in the commit/PR.

Sources & Research

  • Issue #84 (origin) — proposed shape and "why small" framing.
  • crates/core/src/commands/wait.rs (wait_for_text) — poll loop, backoff, persist-on-match, last_error pattern reused by wait_selector.
  • crates/core/src/commands/find.rs (FindQuery, node_matches) — the matcher promoted to query.rs.
  • crates/core/src/commands/snapshot.rs (build, format_result) — cheap build + envelope shape.
  • crates/core/src/commands/wait_timeout.rskind:"wait_timeout" envelope convention (KTD5).
  • src/cli/mod.rs, src/main.rs — global-flag mechanism (--headed) and the single post-dispatch seam (run_with_adapter/finish).