Brings the learning corpus back in line with code that moved this
branch: the gesture-capability and policy docs now describe the
window-level raise in the physical path and link the new abort-state
doc, the reliability contract documents the TIMEOUT kind discriminant
and the wait --action per-name policy variant, the FFI review rule
covers structural repr(C) size drift alongside behavioral parity, the
allocator doc records how the config struct absorbed four more fields
in one place, and the fingerprint doc names the real tri-state decode
error type. Three docs verified accurate with no edits.
Documents the abort-state contract for multi-step physical input (guard
disarm ordering, origin release, end-state suggestions), the three-layer
repr(C) size-pinning discipline born from the AdAction silent-growth
incident, and the named-arms-plus-exhaustiveness-guard pattern for
policy/dispatch mirrors. CONCEPTS.md gains Action Chain and Protected
Process and refreshes Coordinate Fallback with the window-topmost rule.
Three wait tests used a 1ms timeout that can elapse before the loop's
first resolution attempt under load, flaking on machine pressure (the
ambiguous-resolution test needs at least one attempt to record its
observation). 50ms guarantees the first attempt without slowing the
suite.
C11-gated _Static_asserts mirror the Rust-side layout pins so a C
consumer compiling against a drifted header fails at build time, and
the production #[path] prune module carries its privacy rationale.
AgentDeskFixture.swift sat at exactly 400 lines; the scroll card is
fully self-contained (its offset state never leaves the card), so it
moves to FixtureCards.swift as a standalone view with zero bindings,
landing the main file at 374. Harness-facing labels are byte-identical.
DragCanvas gains two intent comments distinguishing the deliberately
empty updateNSView from a forgotten sync.
Names the two TIMEOUT details schemas by their kind discriminant with
the mutated-flag retry rule, points chain-deadline recovery at
AGENT_DESKTOP_CHAIN_TIMEOUT_MS instead of --timeout, extends the
roles_present hint to all non-count selection-mode misses, and aligns
the STALE_REF recovery row with the richer error.rs suggestion.
The CLI-layer guard returned bare INVALID_ARGS while the adapter layer
carried recovery guidance, so agents on the primary surface got 'check
command syntax' for a permanently-disallowed operation and looped on
argument fixes. Both layers now state the same suggestion.
A new ref-action command could ship without a base-policy assertion. A
guard test now scans crates/core/src/commands/ for files calling
context.request( and fails unless each stem appears in the
POLICY_TESTED_COMMANDS list backing the policy assertions.
CGEvents land on the topmost window at the click point, so an app being
frontmost is not enough when the target element lives in a background
window of that app — the physical fallback clicked whatever overlapped
it, and the skip-raise-when-frontmost optimization widened the window
for that. click_via_bounds now raises the element's own AXWindow (AXRaise,
AXMain fallback, brief confirmation poll) via a shared window_ops helper
that focus_window_impl reuses. Verified live: a headed click on a ref in
an occluded Finder window raises that window and lands the click in it.
'docker'.contains('dock') permanently blocked close-app for Docker,
FinderSync-class apps, and anything else embedding a protected name.
Matching is now an exact lowercase name or an exact dot-separated
bundle-id component, so Dock and com.apple.dock stay protected while
Docker, Docker Desktop, FinderSync, and PathFinder stay closable —
pinned by false-positive tests.
A chain deadline truncating the disclosure settle wait returned a plain
step failure, exhausting into ACTION_FAILED — the same masking class
fixed for increments — even though the triggering action may still land
after the truncated wait. Settle exits are now classified: full-budget
misses stay step failures, deadline-truncated waits raise TIMEOUT with
the wanted/observed state, and the poll sleeps are clamped so a tight
deadline still gets at least one read. All TIMEOUT details now carry a
kind discriminant (wait_timeout vs chain_deadline) so agents can branch
without sniffing field names.
Documents the ensured (best-effort, already-frontmost-aware) semantics of
focused:true and its absence-vs-false meaning, the four redaction keys
added in round 2 plus the substring-match behavior, the INTERNAL error
recovery row, the FFI wait-surface asymmetry, the cross-app drag
occlusion caveat, and the AdDragParams/AdAction layout history with the
adjudicated pre-1.0 breaks so fresh reviews stop re-finding them.
The DragCanvas carried two label sources (the NSView and a SwiftUI
modifier on its representable), the same macOS-version-dependent race
the slider/stepper fix removed; the harness-facing label now lives
solely on the AX-actionable NSView.
PointResolveArgs, ResolvedPoint, the ref-or-xy resolver, and
focus_for_physical_input were accumulating in helpers.rs alongside
unrelated ref-action plumbing; they now live in a dedicated
point_resolve module consumed by hover and drag.
The policy mirror lived in a separate helper with a catch-all arm, so a
future action name could silently inherit the headless policy. Parse now
maps every --action name to the exact ActionRequest its real command
runs (type is the only focus-fallback), the catch-all is gone, and a
test pins each name's policy.
helpers_tests (426) splits into resolution/window/pipeline tests, a
ref-action+trace test file, and a shared entry-builder support module.
wait_element_tests (414) splits into predicate-behavior tests and
resolution/lifecycle tests over a widened wait_test_support. wait.rs
(395) loses the element-wait loop to wait_element.rs, and refs_store
(397) moves its tmp-cleanup/retention methods to a refs_store_prune
child module (declared via #[path] so the split keeps base_dir and
snapshots_dir private to the store).
Drag synthesis errors surfaced as bare INTERNAL with no hint about the
gesture's end state. Failures now carry a suggestion stating the button
was released back at the origin (best-effort), no drop was committed,
and where the cursor ends; the guard doc spells out the two best-effort
limits (corrective posts can fail; a self-drop at the origin is a no-op
for most targets).
ensure_app_focused set AXFrontmost unconditionally and reported success
identically whether or not a raise happened; it now no-ops when the app
is already frontmost, so Ok (and the focused:true response field) means
"frontmost ensured" exactly as documented. focus_window_impl gains the
same confirmation poll after its raise, and the poll interval widens to
5ms (10 reads max in the 50ms window).
The guard lived only in the CLI command layer, so ad_close_app could
force-kill session-critical processes (loginwindow, WindowServer, Dock)
that the CLI refuses. close_app_impl now refuses them before any side
effect with the exact CLI error contract, making CLI, FFI, and any future
consumer behave identically; the CLI preflight remains as an earlier
check against the same predicate.
The settle poll could spend 3 x 200ms x 5ms-interval reads (~360 IPCs)
per expand/collapse and overshoot the chain's own deadline. A new
CustomWithDeadline chain step threads the chain deadline into the
disclosure steps, the settle budget is min(200ms, remaining chain
budget), and the interval widens to 20ms (~30 IPCs worst case).
A chain deadline firing mid-increment returned Ok(false), so the step was
recorded as skipped, the chain exhausted into ACTION_FAILED, and the
control sat at a half-applied value the caller could not see — post-state
is only read on success, and ACTION_FAILED recovery guidance points away
from retrying. Expiry is now a TIMEOUT error carrying value_before,
value_at_timeout, target, and a mutated flag in details.
Adds perf: to the allowed commit types (release-please already maps it to
a Performance changelog section), records the pre-1.0 versioning policy so
a BREAKING footer is expected to cut a minor rather than a major, and
shows the optional error.details object in the error envelope docs.
The gitignored local AGENTS.md mirror got the same contract sync.
The harness now fails setup loudly when the fixture build or AX trust is
missing, rebuilds the fixture when sources are newer than the bundle,
asserts hover only against a freshly observed state, and force-collapses
the disclosure so the expand test proves a real flip. The slider/stepper
labels live solely on the NSViews (the AX-actionable elements), removing
the macOS-version-dependent race between two label sources, and the drag
canvas reports a zero frame when detached from a window.
The actionable predicate hardcoded Click, so wait-then-type flows got a
false ready on fields that cannot accept text (the editability check only
runs for editing actions). --action selects click (default), type,
set-value, or clear, and the preflight mirrors each command's real base
policy (type uses its focus-fallback base).
ensure_app_focused slept 50ms per physical input even when the app was
already frontmost; it now polls AXFrontmost (1ms, 50ms deadline).
disclosure_settled slept an unconditional 40ms up to three times per
expand/collapse; it now polls the disclosed state (5ms, 200ms deadline),
converging immediately on fast UIs. The kAXFocusedAttribute settability
probe is gated by role_may_accept_focus, mirroring role_may_bear_value,
and key dispatch reuses ensure_app_focused instead of its inline
duplicate.
AdDragParams is embedded by value in AdAction, so its 8-byte growth grew
the struct C callers pass to ad_execute_action with no size guard —
old-layout callers under-allocate and stack garbage becomes a live
drop_delay_ms. Adds AD_ACTION_SIZE / ad_action_size() with a layout pin,
matching the AdRefEntry pattern.
Also hardens the input boundary: C strings are decoded with a bounded
NUL scan (AD_MAX_STRING_BYTES, sized for CLI argv parity) so a missing
terminator cannot walk arbitrary memory, and the single coarse 1024
array cap becomes published per-field caps (AD_MAX_REF_STATES/ACTIONS/
PATH_DEPTH) with tests just over each limit.
All dispatch sites construct ChainContext with deadline: None, so the
remediation parameter on increment_to_value never received a value and
the 1024-iteration loop ran unbounded by the chain timeout. The chain now
pins its resolved deadline into the context every step observes. Also
extracts the pure write-verification predicates and their tests to
chain_verify.rs, bringing chain.rs back under the 400-LOC limit.
The mouse-up guard disarmed before the final fallible up-event, so a
failed final post left the button held. Worse, its corrective release
fired at the unreached destination, silently committing an aborted drag
as a completed drop (CGEvents resolve at their embedded coordinates).
The guard now owns the release: it disarms only after the up actually
posts, and an early return cancels by dragging back to the origin and
releasing there.
Ref-addressed hover/drag raised the target app unconditionally, violating
the headless no-implicit-focus-steal contract. The point resolver now
returns the owning pid instead of focusing, and commands decide: headless
never raises, --headed raises once (drag focuses only the from-app, fixing
the cross-app double-focus). Responses report focused:true so multi-app
agents can detect the frontmost change.
- SKILL.md: note that the error object may carry an optional `details` (the
actionability report, AMBIGUOUS_TARGET candidates, or a wait TIMEOUT's last
observed state) and that responses should be parsed leniently — `details` and
future fields are additive.
- commands-observation: show the no-match `find` response with the
`roles_present` hint so callers can tell a wrong role name from "none on
screen".
- Extract the reusable fixture components (status readout, native AppKit
slider/stepper, drag canvas, card) into FixtureComponents.swift so each file
is under the 400-LOC limit; build.sh now compiles every .swift in the dir.
- NativeSlider/NativeStepper implement updateNSView so a SwiftUI binding change
syncs back to the NSView.
- The fixture no longer steals focus unconditionally on launch
(activate ignoringOtherApps:false), so it cannot mask headless-policy focus
violations; the harness drives focus explicitly.
- build.sh pins the SDK and a macOS 13 deployment target for reproducible builds.
ActionabilityStatus, ActionabilityCheck, and ActionabilityReport move into their
own files under a new actionability/ module (following the tree/ and actions/
folder pattern); mod.rs keeps the check logic and re-exports the types. Honors
the one-domain-type-per-file rule without changing behavior.
- action_list: gate the AXValue and AXExpanded `is_settable` probes on whether
the role could plausibly carry that capability (unknown roles always probe),
skipping up to two AX round trips per preflight on common click-only targets.
No capability is lost — value/expandable roles still probe.
- resolve_search: reuse one scratch FxHashSet across nodes instead of allocating
a fresh dedup set per node during path and recursive search.
- wait: cap each ref-resolution attempt (750ms) so a slow resolve cannot
consume the whole wait budget on the first poll; the predicate is re-checked
across the full timeout.
- wait: make LatestRefCache timing fields private (no external readers).
- wait: add a unit test for --menu-closed (asserts it waits for open=false).
- close-app: make the graceful and force responses symmetric — both carry
`closed` (force confirms true; graceful cannot confirm, so false) instead of
graceful silently omitting the field.
- close-app: test adapter-error propagation.
- adapter: the default resolve_element_strict_with_timeout now logs that it does
not enforce the deadline, so an adapter that forgets to override it is visible
in traces.
- type: a RAII guard restores the user's clipboard on every scope exit (success,
error, panic) during the paste-based non-ASCII path, shrinking the clobber
window to an unpreventable SIGKILL.
Note: the reviewer's "fail fast on AMBIGUOUS_TARGET with a pinned snapshot"
suggestion is not applied — existing tests prove transient ambiguity resolves on
retry even with a pinned ref (ambiguity is a property of the live tree, not the
refmap), so failing fast would regress intentional, tested behavior.
- Add best-practices/macos-gesture-headless-capability: which desktop gestures
have a headless AX path on macOS (double-click via AXOpen; triple-click/hover/
drag are physical-only; SwiftUI controls vs native AppKit), and why the command
never decides — the platform adapter owns headless-vs-physical.
- Refresh two policy learnings for the physical->headed rename: ActionRequest::
physical -> headed and AD_POLICY_KIND_PHYSICAL -> AD_POLICY_KIND_HEADED, noting
the new global --headed upgrade path via CommandContext::request.
- CLAUDE.md: remove ~130 lines of reference material that duplicated code,
Cargo.toml, or the skills (full PlatformAdapter trait dump, Key Types listing,
macOS API listings, dependency/build-config tables, the 54-command table).
Replaced the stale trait dump with a pointer to adapter.rs (which also fixes
the review's stale-trait-docs finding) and kept only the non-obvious gotchas.
CLAUDE.md is now standards, invariants, and conventions.
- Document, on macOS (Phase 1), which gestures have a headless path: most ref
actions do; double-click via AXOpen; triple-click/hover/drag are cursor
gestures with no AX equivalent (physical only). The command surface is
platform-agnostic — a future Windows/Linux adapter that exposes a headless
path lights it up with no command or core change. Added to README and the
interaction reference.
- FFI skill: AD_POLICY_KIND_PHYSICAL is now AD_POLICY_KIND_HEADED.
Closes the honesty gaps the review found and covers the interactions that were
missing, all verified by independent before/after observation:
- twins: the fixture twins now record distinct effects (twin-a/twin-b) and the
assert requires the ADDRESSED twin to fire (or AMBIGUOUS_TARGET), instead of
passing on any ok:true.
- click: click-status is a counter, so the headed pass must observe a fresh
increment rather than inheriting the headless pass's value.
- adds triple-click + hover (headed gestures), tab selection (TabView tabs are
radiobuttons), context-menu open + item selection, and menu-bar enumeration
via --surface menubar.
- adds a performance section reporting per-command CLI wall-clock (snapshot,
find, get, click, set-value, type) with a soft <2s snapshot gate.
- documents the SwiftUI CommandMenu and cross-app drop limitations as tracked
notes, not silent skips.
AdDragParams gained a drop_delay_ms field but, unlike AdRefEntry, had no size
guard — an old caller's smaller allocation would let Rust read past it and turn
stack garbage into a real drop delay. Add AD_DRAG_PARAMS_SIZE, ad_drag_params_size(),
a compile-time layout assertion, and a zero-init note, matching the ref-entry
pattern.
This branch makes several consumer-visible contract changes that release
tooling must cut as a major. They are gathered here because the release workflow
ships the C header as an artifact.
BREAKING CHANGE: the C ABI and CLI/JSON contract changed on this branch.
- AdPolicyKind: AD_POLICY_KIND_PHYSICAL is renamed AD_POLICY_KIND_HEADED
(discriminant 2 unchanged, so compiled binaries are safe; source-level C
consumers must rename). No back-compat alias is kept — "physical" is gone.
- AdRefEntry grew (caller-allocated input); validate layout with
AD_REF_ENTRY_SIZE / ad_ref_entry_size().
- AdDragParams grew; validate with AD_DRAG_PARAMS_SIZE / ad_drag_params_size()
and zero-initialize before use.
- close-app graceful response no longer includes closed:true; it returns
{ method: "graceful", requested: true } because a graceful quit cannot be
synchronously confirmed.
Ref-action physical fallbacks (click/scroll/type) already brought the target
app frontmost before synthesizing CGEvents. The raw-input commands (drag, hover,
mouse-*) resolving a point from a ref did not — the resolver had the pid but
discarded it, so the adapter saw only coordinates and synthetic events could
land on whatever window happened to be frontmost.
Add a best-effort focus_app(pid) to PlatformAdapter (macOS uses
ensure_app_focused; other adapters default to not_supported). The point resolver
now focuses the ref's app before returning, so every physical interaction that
targets a known element raises its window first. Coordinate (--xy) input is
unchanged: the caller owns the target there.
