mirror of https://github.com/lahfir/agent-desktop.git synced 2026-07-31 03:09:21 +00:00

History

Lahfir c29b9dcf3a fix(review): eliminate per-element AX round trips and close remaining review findings Fold AXPosition/AXSize and the scrollbar probe into the existing AXUIElementCopyMultipleAttributeValues batch so tree traversal and the actionability preflight pay one IPC per element instead of up to four; A/B benchmark shows strictly-faster snapshots with identical ref counts and scroll capabilities (Finder 4.5s -> 2.4s same-session, Docker Desktop parity at 440 refs / 933 scroll-capable nodes). Consolidate the CLI and FFI ref-action pipelines into one core execute_resolved path (actionability, tracing, and dispatch semantics live once; FFI passes a default context). Remove the no-context execute() shims from is/right-click/snapshot/wait and the test-only helper shims; every command now takes an explicit CommandContext. Split the macOS resolver into resolve (orchestration), resolve_search (candidate collection), and resolve_classify (strict classification), clearing the 400-LOC ceiling with room to grow. Notification waits now retry transient baseline failures inside the timeout budget with the same retryable gate as window/text waits and report last_error in timeout details instead of aborting on the first flake; a baseline is never fabricated. Refmap writes clean up their temp file on failure, stale *.tmp orphans are swept under the store lock, and save_existing_snapshot re-verifies snapshot ownership inside the owning store's write lock with bounded re-discovery before deterministically recreating in the caller's store. Coverage hardening: zero-budget wait timeout shape, wait --text --count 0 absence detection, ref-action pipeline call-count guard (1 resolve / 1 live read / 1 dispatch), duplicate snapshot-id collision on load, pruned-everywhere recreation, tmp sweep and rename-failure cleanup, FFI AMBIGUOUS_TARGET last-error code assertion.	2026-06-17 13:43:19 -07:00
..
README.md	fix: make explicit snapshots session-independent	2026-06-17 13:43:19 -07:00
ref_action_contract.rs	fix(review): eliminate per-element AX round trips and close remaining review findings	2026-06-17 13:43:19 -07:00

Lahfir c29b9dcf3a fix(review): eliminate per-element AX round trips and close remaining review findings

Fold AXPosition/AXSize and the scrollbar probe into the existing
AXUIElementCopyMultipleAttributeValues batch so tree traversal and the
actionability preflight pay one IPC per element instead of up to four;
A/B benchmark shows strictly-faster snapshots with identical ref counts
and scroll capabilities (Finder 4.5s -> 2.4s same-session, Docker
Desktop parity at 440 refs / 933 scroll-capable nodes).

Consolidate the CLI and FFI ref-action pipelines into one core
execute_resolved path (actionability, tracing, and dispatch semantics
live once; FFI passes a default context). Remove the no-context
execute() shims from is/right-click/snapshot/wait and the test-only
helper shims; every command now takes an explicit CommandContext.

Split the macOS resolver into resolve (orchestration), resolve_search
(candidate collection), and resolve_classify (strict classification),
clearing the 400-LOC ceiling with room to grow.

Notification waits now retry transient baseline failures inside the
timeout budget with the same retryable gate as window/text waits and
report last_error in timeout details instead of aborting on the first
flake; a baseline is never fabricated.

Refmap writes clean up their temp file on failure, stale *.tmp orphans
are swept under the store lock, and save_existing_snapshot re-verifies
snapshot ownership inside the owning store's write lock with bounded
re-discovery before deterministically recreating in the caller's store.

Coverage hardening: zero-budget wait timeout shape, wait --text
--count 0 absence detection, ref-action pipeline call-count guard
(1 resolve / 1 live read / 1 dispatch), duplicate snapshot-id collision
on load, pruned-everywhere recreation, tmp sweep and rename-failure
cleanup, FFI AMBIGUOUS_TARGET last-error code assertion.

2026-06-17 13:43:19 -07:00

README.md

fix: make explicit snapshots session-independent

2026-06-17 13:43:19 -07:00

ref_action_contract.rs

fix(review): eliminate per-element AX round trips and close remaining review findings

2026-06-17 13:43:19 -07:00

README.md

Adapter Reliability Conformance

Every platform adapter must satisfy the same ref/action contract. The macOS adapter is the first implementation, but the tests are written against PlatformAdapter semantics so Windows UIA and Linux AT-SPI can reuse the same expectations.

The reusable command-path helper lives in tests/conformance/ref_action_contract.rs. The executable smoke harness in src/tests/conformance.rs uses it with a mock adapter; future Windows and Linux fixtures can call the same helper with real adapter-provided refs.

Required Gates

Area	Required behavior
Snapshot refs	Refs are depth-first, snapshot-scoped, and explicit snapshot IDs resolve directly
Strict resolve	A ref resolves only when identity still matches; stale refs return `STALE_REF`
Ambiguity	Multiple plausible matches return `AMBIGUOUS_TARGET`, never an arbitrary click
Actionability	Ref actions check live visibility, stability, enabled state, supported action, policy, and editability before dispatch
Wait recovery	`wait --element` can poll the latest session refmap when no snapshot is pinned, honors the caller timeout while resolving, and reports the last observed predicate state
Session latest scope	Commands that omit `--snapshot` read and write only the active session's latest refmap
Explicit snapshot scope	Passing `--snapshot <id>` resolves that pinned snapshot even when the caller omits the original session
Trace	`--trace <path>` writes JSONL diagnostics outside stdout and is best-effort unless strict
FFI parity	FFI ref actions use strict resolve and actionability checks before adapter dispatch

Platform Matrix

Fixture	macOS AX	Windows UIA	Linux AT-SPI
Two identical buttons produce ambiguity	Required	Required	Required
Ref disappears after snapshot	Required	Required	Required
Disabled button blocks click before dispatch	Required	Required	Required
Text field supports value/type actionability	Required	Required	Required
Session A latest refmap is invisible to Session B	Required	Required	Required
Batch item session overrides inherited session	Required	Required	Required
FFI `AdRefEntry` preserves full identity envelope	Required	Required	Required

Adding Windows or Linux

Add adapter-specific integration fixtures, but keep expected errors and JSON shapes identical. Prefer semantic platform actions first (AXPress, UIA Invoke/Value/Selection patterns, AT-SPI actions). Coordinate input is a lower confidence fallback and must remain explicit in policy or command choice.