mirror of https://github.com/lahfir/agent-desktop.git synced 2026-07-26 17:12:15 +00:00

feat!: implement Playwright-grade foundation contract

Settle the Playwright-grade reliability contract in agent-desktop-core
before the Windows/Linux adapters are built, so they inherit it instead
of redesigning it. Every command now observes, waits, verifies, and
reports honestly instead of firing blindly.

Highlights: capability-supertrait split of PlatformAdapter with
not_supported() defaults; canonical role/state vocabulary with live
`is --property visible`; display enumeration (`list-displays`) and honest
`--screen` with scale factor; truthful Automation permission; `native_id`
identity spine; window-id-first resolution; serializable `LocatorQuery`
with live `find`; default-on auto-wait before every ref action; three-way
`hit_test` occlusion gate; `scroll_into_view` in core; core accessible-name
precedence; typed `ActionStep` delivery tier; `ProcessState` and
`APP_UNRESPONSIVE`; `LaunchOptions`; baseline-diff desktop signals
(`wait --event`); typed clipboard (`Text`/`Image`/`FileUrls`); mouse
modifier chords and `mouse-wheel`. Hardened through a 35-reviewer pass with
independent validation and a green live e2e gate (109/0), plus a
head-vs-main performance comparison harness.

BREAKING CHANGE: default-on auto-wait changes the timing of every
previously-untouched ref-action call (bounded 5000 ms default; `--timeout-ms 0`
restores single-shot). `ENVELOPE_VERSION` is now `2.1` (adds the
`APP_UNRESPONSIVE` code and process state in error details). FFI ABI major
is `3` (append-only struct evolution; `wait --event` is intentionally not
exposed over FFI). The legacy string clipboard API is removed in favor of
typed content. `key-down`/`key-up` fail closed until daemon-owned held input
exists. `close-app` verifies termination and the osascript fallback path is
removed. `--text` matching is subtree containment: `find --text X --first`
returns the outermost matching container.

2026-07-20 00:21:38 -07:00

2.3 KiB

Raw Blame History

title

date

Keep ref allocation in one recursive owner

Context

Full snapshots and rooted drill-downs both turn an observed tree into refs. They differ only in their allocation options, source evidence, and scope. Two recursive implementations would inevitably make the same UI allocate different refs depending on how it was observed.

Guidance

allocate_refs in crates/core/src/ref_alloc.rs is the only recursive allocator. Callers supply a RefAllocConfig composed of RefAllocOptions, RefAllocSource, and RefAllocScope rather than adding another allocator or a positional argument list.

The allocator owns these semantics together:

ref eligibility: interactive roles or primary advertised actions, never a bare focus affordance;
identity, geometry, capability, source, and scope evidence in RefEntry;
bounds omission, compact collapse, and interactive-only pruning;
named skeleton anchors that are legitimate drill-down targets.

Full snapshots use an empty root scope. Drill-down passes the root ref and its path prefix, then updates only that root's descendants in the stored map.

Why This Matters

Ref allocation is an identity boundary, not presentation formatting. A change to one path but not the other can create stale refs, lose scope evidence, or make an action available in one observation mode but impossible in another.

Prevention

Add behavior once in ref_alloc.rs and cover both full and rooted snapshots.
Prefer a nested config type when a new allocation dimension appears; do not add _with_root or _for_skeleton copies.
Treat a new recursive allocator as an architecture violation unless its output contract is genuinely different and documented.

2.3 KiB Raw Blame History