Extracts a single resolve_within_deadline poll-resolve helper shared by
ref_action_wait's auto-wait loop and wait_element's element wait loop
(F13), so the 750ms per-attempt cap and deadline math has one owner;
each caller keeps its own retry classification and side effects
(LatestRefCache refresh, ElementNotFound retry) as before.
Deletes classify_query_result/ambiguous_candidate_summaries/
QueryCandidateSummary from locator.rs (zero callers) and simplifies
find.rs's materialize_match to its snapshot-clone, dropping the
index/tree/query params it silently discarded. Chosen over wiring the
live path to build responses from live handles, which would change
find's --count/--last semantics.
Role::is_interactive now delegates to roles::INTERACTIVE_ROLES instead
of re-encoding the 16 role names (F22).
Consolidates three byte-identical NoopAdapter/StubSystemOps blanket
test doubles (src/batch/tests.rs, src/dispatch/notifications.rs,
tests/conformance/window_identity_contract.rs) into one file at
tests/support/noop_ops.rs, included via #[path] since the binary's own
unit tests and the standalone conformance integration crate cannot
share a Rust module across the crate boundary.
Extracts locate_verified_record in window_resolve.rs (parse -> find ->
verify, previously duplicated by resolve_window_strict and
resolve_window_element_strict) and compute_readonly in element.rs
(previously duplicated identically by fetch_node_attrs and
fetch_node_attrs_slow).
Hoists element.rs's 18-entry CFString attribute-name array behind a
thread-local cache (CFString/CFArray aren't Send+Sync so a process-wide
LazyLock isn't viable) instead of rebuilding it on every node. Makes
query.rs's collect_matches short-circuit on the cheap role check before
building live-tree state/children context for elements that can never
match.
Replace the tautological state-vocabulary conformance tests (they rebuilt
their expected set from the same state:: constants they checked, so they
could never fail) with real guards: a source-scan test in core that reads
the actual state-consuming call sites and flags bare literals that bypass
state::, and a macOS test that drives the real states_from_element producer
over representative inputs and asserts the emitted tokens are a subset of
STATE_VOCABULARY. Both are paired with a should_panic test proving
assert_states_in_vocabulary genuinely rejects a bogus token.
Migrate the live "disabled" string literal in
actionability::states_are_enabled to state::DISABLED, and document the
three vocabulary-only tokens (invalid/multiselectable/haspopup) that have
no macOS AX producer today.
Thread real window bounds into the two macOS call sites that build
StateReaderContext with window_bounds hardcoded to None (post_state.rs's
post-action/live state reads and query.rs's find/query tree walk), so the
offscreen token can actually be computed there instead of being silently
unreachable. post_state.rs resolves the owning window via the AXWindow
attribute; query.rs inherits window bounds down the recursive walk the same
way builder.rs does, capturing a node's own bounds once its role is
AXWindow.
Add an ObservationOps call (resolve_element_strict) to the U0 capability
conformance test's exercise() helper, which previously called zero
ObservationOps methods and would not have caught a break in that
supertrait.
accname.rs previously reduced NameEvidence with a 2-way title-or-description
fallback despite the trait already exposing 7 evidence fields. Implement the
documented KTD6 precedence (explicit label -> labelled-by text -> native
title -> static-role value -> aggregated child label -> placeholder ->
description last) as compute_name/compute_description over NameEvidence,
plus a join_child_labels aggregation primitive.
Migrate macOS's resolve_element_name to a thin wrapper that gathers raw
NameEvidence (name_evidence_impl) and reduces it via core's compute_name,
removing the inline fallback chain it used to own. name_evidence_impl now
reads AXTitleUIElement (labelled-by), AXPlaceholderValue, and aggregates
multiple child labels via join_child_labels instead of returning only the
first match. resolve_search.rs, resolve_classify.rs, and chain_menu_steps.rs
need no changes: they already consume resolve_element_name's return value
and now get the correct precedence for free.
open_session silently returned Ok(None) instead of following the
blanket not_supported convention every other adapter default uses,
AdapterSession had lost its Sync bound, and the SessionAffinity
parameter the plan specified never shipped. Introduces
session_affinity.rs (SessionAffinity { session_id }), restores
AdapterSession: Send + Sync, and changes open_session's signature to
open_session(&SessionAffinity) -> Result<Box<dyn AdapterSession>,
AdapterError> with a default Err(not_supported("open_session")).
native_id was inserted mid-struct in AdRefEntry (offset 40), shifting every
field from `states` onward and silently corrupting a prebuilt C consumer's
field reads. Move it to the end, matching AdActionStep's append-only
evolution pattern, and bump AD_ABI_VERSION_MAJOR so a consumer built against
the old layout fails ad_init instead of misreading memory.
Also collapse execute_by_ref_timeout.rs, which hand-duplicated ~85 lines of
the @generated ad_execute_by_ref body, into a proper Family-B codegen
template so it stays in lockstep with the canonical wrapper instead of
drifting from it.
- crates/ffi/src/types/ref_entry.rs: move native_id to the end of AdRefEntry
- crates/ffi/src/abi_version.rs: bump AD_ABI_VERSION_MAJOR 1 -> 2, +regression
test pinning the bump so a revert fails a real assertion
- crates/ffi/tests/c_abi_layout.rs: explicit per-field offset_of! asserts for
every AdRefEntry field (not just monotonic ordering) + zeroed-read sentinel
now asserts native_id.is_null()
- crates/ffi/codegen_templates/execute_by_ref_timeout.rs.in +
crates/ffi/build.rs + tests/codegen_exhaustiveness.rs: generate
ad_execute_by_ref_timeout through the same Family-B pipeline as
ad_execute_by_ref instead of a hand-maintained duplicate
- crates/ffi/include/agent_desktop.h: regenerated via
scripts/update-ffi-header.sh (cbindgen 0.29.4)
Modifiers were a silent no-op: MouseEvent already carried a modifiers
field but synthesize_mouse never read it, mouse_wheel dropped its
_modifiers param outright, and mouse-click/mouse-down/mouse-up had no
--modifiers flag to even request a chord.
- macos/input/mouse.rs: derive CGEventFlags from the requested modifier
chord and set it explicitly on every synthesized CGEvent (including
the empty case), so no ambient or prior-call flags can leak onto an
unmodified click; thread modifiers through synthesize_scroll_at and
set them on the wheel event too.
- macos/adapter.rs: stop dropping mouse_wheel's modifiers parameter.
- cli_args/actions.rs + dispatch: add --modifiers to mouse-click,
mouse-down, and mouse-up (mirroring the existing mouse-wheel flag)
and thread it through parse_modifiers.
- core mouse_click/mouse_down/mouse_up commands: stop hardcoding
modifiers: Vec::new() and pass the requested chord through.
- ffi/input/mouse.rs: add the additive ad_mouse_event_with_modifiers
entrypoint (modifiers as an array + count, mirroring AdKeyCombo) so
FFI callers can reach chorded clicks without changing the pinned
AdMouseEvent layout; regenerate the committed header.
--args was emitted once per --arg, handing the launched app a stray
--args token instead of its intended argument; argv assembly is now a
pure, unit-tested fn that emits --args exactly once. --no-attach was
silently ignored: launch_app_with_options_impl always attached-or-waited
regardless of the flag. It now fails with a structured error naming the
running pid when the app is already running, and returns immediately
without the window-wait loop when it is not. The raw app id no longer
lands in trace-reachable error messages; it travels only in
details.app_name, which redacts on trace export.
Split the launch path out of app_ops.rs into a new launch.rs sibling
module (app_ops.rs would otherwise exceed the 400-line cap).
Replaces the flat {format,text,bytes_base64} ClipboardContent struct with
the plan's ClipboardContent { Text, Image, FileUrls } enum and removes the
string-only get_clipboard/set_clipboard adapter methods (KTD13: remove, not
wrap) in favor of get_clipboard_content/set_clipboard_content as the sole
surface, keeping clear_clipboard unchanged.
macOS now round-trips text, PNG images, and public.file-url references
through NSPasteboard instead of returning not_supported for image/file-urls.
clipboard-get defaults to text when --format is omitted, writes image bytes
to an --out path (or a private 0600/O_NOFOLLOW/atomic temp file under the
session dir when --out is omitted, since clipboard images can carry copied
secrets), and reports {type,path,width,height}. clipboard-set gains --image
and repeatable --file-url inputs; missing file-url paths report only a
count and entry index, never the path content, since that can reach traces.
FFI's ad_get_clipboard/ad_set_clipboard now delegate through the content
API with unchanged C signatures.
Replace the 3-kind DesktopSignal vocabulary with a full EventKind set
(WindowOpened, WindowClosed, AppLaunched, AppTerminated,
FocusChangedWindow, SurfaceAppeared, SurfaceDismissed) and a pure,
adapter-double-testable diff_signals(baseline, current) over an
id/pid-keyed SignalBaseline. wait --event window-opened/app-launched/
surface-appeared now works from just --app, without the caller naming
a window id or title up front (R16/AE6) — wait_event.rs captures a
baseline at wait start and polls diff_signals against fresh captures,
same shape as the existing notification baseline loop.
Folding window/app identity into id- and pid-keyed sets (rather than
count comparisons or id-or-title matching) closes two real bugs: a
title-matching window could win a window_focused wait for the wrong
id, and a concurrent unrelated window open could mask a window_closed
event gated on window_count. Event timeouts now carry
details.kind == "wait_timeout" like every sibling wait mode.
--window alongside --event now narrows the event wait to a window
title instead of tripping the "exactly one mode" validator.
Reshape ProcessState to the plan's {Running, Exited{code}, Crashed,
Unresponsive} contract instead of the ad hoc {Responsive, Unresponsive,
Unknown}. macOS now does a kill(pid,0)-style liveness check before
probing AX at all, and only classifies Unresponsive after a second
consecutive kAXErrorCannotComplete, so one transient AX blip on a
healthy-but-busy app no longer hard-fails the action. The classification
threshold is isolated behind a pure, platform-independent classify() fn.
ensure_process_responsive was an unconditional preflight hard-gate that
could turn a would-succeed action into a failure and propagated a
transient probe error as the terminal error. Replace it with
enrich_with_process_state: best-effort, terminal-only enrichment that
runs exactly once (never per auto-wait tick), never converts a success
into a failure, attaches details.process_state on STALE_REF/APP_NOT_FOUND,
and only surfaces APP_UNRESPONSIVE when the process is genuinely
classified Unresponsive.
Add the missing retry_token_for_code arm for AppUnresponsive.
HitTestResult becomes ReachesTarget | InterceptedBy { role, name, bounds }
| Unknown instead of a boolean, so a probe failure or a hit on the
target's own ancestor (composited/custom-drawn containers) is never
reported as a false occlusion Fail. receives_events_check maps the
three states to Pass/Fail/Unknown and carries the occluder's role and
redactable name in ActionabilityCheck.occluder instead of a hardcoded
string. requires_hit_test() now covers Drag. visibility_check also
fails on the HIDDEN/OFFSCREEN state vocabulary, not just zero bounds.
macOS hit_test_impl reuses the bounded ax_helpers::try_each_ancestor
walk instead of a hand-rolled unbounded AXParent loop, and treats
every probe failure (null/zero bounds, missing pid, AX error) as
Unknown rather than a false Fail.
hover --ref and drag --from/--to previously resolved a ref straight to
a center point via point_resolve without ever consulting hit_test, so
an occluded target dispatched blind. resolve_point_from_ref_or_xy_with_context
now runs the new receives_events-only check on the ref-targeted path
before returning the point; raw --xy input is unaffected by design.
type, set-value, select, scroll, hover, and drag were the only 6 of 18
ref-addressed commands without a --timeout-ms flag: their core Args
structs hardcoded RefArgs.timeout_ms to None, so auto-wait retry
(landed for click and friends) silently never applied to them. Add the
paired clap/serde --timeout-ms default (5000, matching RefArgs) to
TypeArgs/SetValueArgs/SelectArgs/ScrollArgs/HoverArgs/DragCliArgs and
thread the normalized value through dispatch into the core command
structs. hover/drag resolve coordinates rather than dispatching an
action, so they gain a dedicated resolve-retry helper
(helpers::resolve_point_with_wait) that retries a transient
STALE_REF/AMBIGUOUS_TARGET/TIMEOUT within the same budget.
Also fix ref_action_wait::execute_poll_loop double-checking
actionability: it ran actionability::check_live itself and then handed
off to dispatch (execute_resolved), which runs check_live again. A
failure on that second, redundant check propagated via a bare `?` that
bypassed the retry/permanent classification entirely, so a transient
actionability flip between the two checks failed the action outright
instead of retrying. The loop now calls dispatch directly per
iteration and classifies whatever it returns, so there is exactly one
check_live per attempt and every actionability failure goes through
retry/permanent classification. Single-shot (timeout_ms: None) is
unaffected — it already only checked once.
Also drop the dead `timeout_ms: None` literal in execute_by_ref's
ActionRequest construction (it was always overwritten by
execute_ref_action_with_context) in favor of the actual normalized
value, so the struct literal doesn't misrepresent what ships.
An unbounded --timeout-ms flowed into Instant::now() + Duration::from_millis(ms)
and panicked on overflow; with no catch_unwind on the CLI path this was a raw
crash on trivial input (e.g. --timeout-ms 99999999999999999999). Clamp the
budget to a 24h ceiling before deadline construction, with a regression test.
Also add APP_UNRESPONSIVE to the error-code as_str/serde consistency test, which
was silently non-exhaustive after the 16th variant landed.
The --env KEY=VALUE parser interpolated the raw pair (including the secret
value) into INVALID_ARGS messages, which reach the unredacted `message` field
of command.end trace events (redaction is a field-name allowlist, not a
content scanner). Report the entry index and rule only, never the value, per
docs/solutions/conventions/keep-raw-arguments-out-of-trace-reachable-error-messages.md,
with a regression test asserting a secret marker never appears in the message.
scroll_into_view reused the hand-rolled AXScrollToVisible CFString+perform
sequence that ax_helpers::try_ax_action already provides.
Scroll offscreen ref targets into view before actions, add accname
NameEvidence and supported_surfaces in status, gate on ProcessState with
APP_UNRESPONSIVE and envelope 2.1, extend launch with args/env/cwd,
wire desktop signal waits via --event, typed clipboard formats, and
mouse-wheel with modifier support.
Co-authored-by: Cursor <cursoragent@cursor.com>
Expose ObservationOps::hit_test with macOS AX element-at-position probing
and fail pointer-targeting ref actions when the center point is occluded.
Co-authored-by: Cursor <cursoragent@cursor.com>
Poll resolve and actionability until the budget expires so transient stale
refs settle before dispatch; CLI and FFI default to 5000 ms with --timeout-ms
and ad_execute_by_ref_timeout for overrides.
BREAKING CHANGE: ref actions now auto-wait up to 5000 ms by default; pass
--timeout-ms 0 or ad_execute_by_ref_timeout(..., 0, ...) to restore
single-shot behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce a serializable LocatorQuery contract with core matching and
classify_query_result, wire find through resolve_query on macOS with
snapshot fallback for stub adapters, and add --exact, --state, --native-id,
and --description CLI flags.
Co-authored-by: Cursor <cursoragent@cursor.com>
Window ids are now the primary lookup key for macOS snapshot, focus, and
window-op paths, with pid/title corroboration to fail closed on recycled
CGWindow numbers instead of silently matching the first same-titled window.
Co-authored-by: Cursor <cursoragent@cursor.com>
Capture AXIdentifier as native_id with auto-generated filtering, prioritize it
in identity_matches, and extend AdRefEntry via the size-pin sequence.
Co-authored-by: Cursor <cursoragent@cursor.com>
Complete the display capture contract with enumerable displays, scale_factor
on captures, and INVALID_ARGS when --screen is out of range.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add a shared state_reader for tree and live reads, expand AX batch
attributes, and emit hidden/offscreen/indeterminate tokens from evidence.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add role/state vocabulary modules and move is --property visible onto live
bounds plus hidden/offscreen tokens so off-screen elements no longer pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
Restructure the 397-LOC adapter contract into ObservationOps, ActionOps,
InputOps, and SystemOps with a blanket composed trait so platform adapters
and test doubles gain file-budget headroom without changing behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds the trace read/replay layer on the session-first foundation: `trace show` merges per-process segments into one deterministic timeline (bounded JSON for agents), and `trace export` renders a single self-contained, XSS-safe HTML viewer for humans. Opt-in `session start --screenshots` captures pre/post-action screenshots and refmap copies; command.start/end boundary events and a versioned trace.meta header make a step-by-step replay reconstructable. Redaction is hardened so raw caller arguments never leak into trace-reachable error messages. Available across CLI, batch, and FFI.
Sessions now own tracing: `session start` creates a manifest-gated trace sink with per-process JSONL segments under ~/.agent-desktop/sessions/<id>/trace/, so agents set the session once instead of passing --trace on every command. Bare --session stays snapshot-namespace-only; --trace <path> still overrides for CI and one-offs.
Adds session start/end/list/gc, activation resolution (flag > AGENT_DESKTOP_SESSION > pointer), FFI verification tests, and status fields for session_id + tracing.
Global --wait-for / --wait-for-gone (-w) and --wait-timeout flags that poll the accessibility tree until a role:text selector appears or disappears, then return the snapshot. Honored by snapshot and the 16 ref-action commands; post-action waits scope to the acted-on window. Closes#84.
Hardens the macOS adapter and platform-neutral core: stuck-key/clipboard/scroll/click fixes, deep-copy clipboard restore, notification-session and resolve-deadline hardening, blocked-combo guardrail moved to the adapter with a --force caller override, plus ~60 behavior-pinning tests. No breaking changes.
Make the retained_handle null check unconditional so release builds no longer risk CFRetain(null); tests keep benign placeholder behavior, production returns ElementNotFound.
Completes the FFI plan: U9 adds a Python ctypes smoke harness proving the C ABI from a non-Rust host; U10 adds cross-platform parity CI gates for header drift, codegen drift, panic-unwind, and stub-adapter passthrough; U11 generates the command-backed wrappers from build.rs templates with a committed drift-gated output. The C ABI stays byte-identical and the surface is Windows and Linux ready with no new FFI code.
* build: make cbindgen regenerate the C ABI static-assert guards
Add [const] allow_static_const = false to cbindgen.toml so every
pub const AD_*_SIZE emits as a #define macro (a C constant expression)
rather than static const (which is not valid inside _Static_assert).
Add a cbindgen.toml trailer with all 19 C11 ABI guards (6 sizeof, 6
_Alignof, 7 offsetof). Each sizeof guard references the corresponding
AD_*_SIZE macro so the size literal lives in exactly one place: the
Rust source. The Rust source already has compile-time asserts tying each
const to its struct, so the chain is: Rust const → #define macro →
_Static_assert → C compile gate.
Rename pub const MAX_C_STRING_BYTES to AD_MAX_STRING_BYTES so the
public ABI name matches the header macro (#define AD_MAX_STRING_BYTES)
and the internal const name no longer leaks. Update all call sites and
the describe() error string.
Pin cbindgen 0.29.4 in scripts/update-ffi-header.sh so regen is
reproducible.
* test: stabilize flaky wait-resolution test under CI load
element_wait_retries_transient_ambiguous_resolution and its sibling used a
250ms budget that the happy path clears instantly but a loaded CI runner
(full --lib --workspace in parallel) can exceed before the transient
AmbiguousTarget retry resolves — an intermittent red CI unrelated to any
code change. Raise the ceiling to 2000ms; the success path still returns on
the second resolve, so the test stays fast while becoming load-robust.
* refactor: enforce cbindgen version in update-ffi-header.sh
The script documented 0.29.4 as required but accepted any installed
version silently. A mismatched cbindgen can produce a semantically
different header with no diagnostic. Fail fast with a clear install
command if the version does not match exactly.
* feat: add ad_abi_version and ad_init ABI handshake
* refactor: address review on ad_abi_version
* feat: add ad_version FFI entrypoint
* refactor: move guard_non_null inside trap_panic for convention parity
All other FFI entrypoints validate input pointers as the first statement
inside trap_panic(|| unsafe { … }). ad_version placed the guard outside
the closure; align it with the uniform convention throughout the crate.
* feat: add session-scoped FFI adapter constructor
* refactor: address review on session context
* feat: add ad_set_log_callback with tracing layer
* refactor: address review on log callback
- Tighten core visibility: revert `pub mod trace` to `pub(crate) mod trace`,
re-export only `sanitize_trace_value` at the crate root so the FFI crate
imports it without exposing `TraceConfig` and internal helpers publicly
- Add per-thread re-entrancy guard (`IN_CALLBACK` + `CallbackGuard` RAII
drop) to `on_event`: a consumer callback that emits `tracing` events now
silently drops the recursive invocation instead of risking a stack overflow
that `catch_unwind` cannot stop
- Update module doc and `ad_set_log_callback` `///` doc to document the
re-entrancy protection; regenerate committed FFI header via cbindgen
* feat: add ad_snapshot with refmap pipeline
* refactor: address review on ad_snapshot
Promote app_error_to_adapter to pub(crate) in commands/mod.rs so future
command files share one conversion instead of copying the match per file.
Document the tri-state *out contract (null on arg/infra errors, populated
JSON envelope on command-level errors) in both the Rust doc comment and
the C header.
* feat: add ad_status FFI entrypoint
* refactor: address review on ad_status
* feat: add ad_wait with size-pinned AdWaitArgs
* refactor: DRY ad_wait field decoding
* fix: align command_context error return with errno-style last-error invariant
The context failure arm in ad_wait hardcoded ErrInternal while
set_last_error stored the actual mapped code; callers relying on
ad_last_error_code() == return value would see a mismatch. Mirror the
execute() Err arm and return last_error_code() instead.
* feat: add ad_execute_by_ref with strict resolution
* refactor: address review on ad_execute_by_ref
- extract app_error_to_adapter to error.rs pub(crate), removing
duplicate copies from snapshot.rs and execute_by_ref.rs
- replace 24-line tri-state ref_id decode with required_adapter_string
- use last_error_code() on validate_ref_id failure path for consistency
- fix *out dual-channel doc (null on guard/decode failure, non-null
JSON envelope on command-level errors) in both Rust doc and C header
- rename misleading test: stale_ref_returns_error → returns_error_envelope
(missing refmap surfaces SNAPSHOT_NOT_FOUND, not STALE_REF)
* docs: restore ABI header comments lost in cbindgen regen
Add /// docs on the Rust source so cbindgen re-emits the privacy note on
ad_last_error_details, the AdResult forward-compat note, and the behavioral
descriptions for the ad_execute_action* family.
* refactor: dedup ad_wait error conversion via shared helper
Replace wait.rs's local app_error_to_adapter_error with the shared
commands::app_error_to_adapter, giving one canonical AppError->AdapterError
conversion across every ffi command.
* fix: harden ABI assert trailer against double-include, document panic guard
Add a one-shot #ifndef AGENT_DESKTOP_ABI_ASSERTS guard around the
_Static_assert trailer in cbindgen.toml so re-including agent_desktop.h
in a single translation unit is unambiguously safe in all C standards
(C11 already allows repeated file-scope _Static_assert, but the guard
makes the property explicit and unconditional).
Add c_header_double_include.rs regression test that includes the committed
header twice via the system cc and asserts the compile succeeds, mirroring
the existing c_header_compile.rs harness pattern.
For Finding #14 (panic=abort guard): empirical probe confirms that Cargo
always sets CARGO_CFG_PANIC="unwind" in the build script for this crate
regardless of the active profile's panic setting — because the crate
declares both cdylib and rlib crate-types. A build-time check would be
silently inert. Document the invariant and the infeasibility of the
CARGO_CFG_PANIC guard in build.rs module docs instead; the release-ffi
profile enforces unwind at the profile level.
* fix(ffi): surface foreign-subscriber conflict, soften ad_init wording, direct agents to ad_snapshot
Finding #15: replace discarded try_init result with OnceLock<bool> so the first
ad_set_log_callback call with a non-null callback returns ErrInternal when a
foreign global subscriber already owns the process, rather than silently no-oping.
Re-registration and NULL-unregister paths are unaffected. Adds three unit tests
covering the new routing logic.
Finding #8: soften ad_init and AD_ABI_VERSION_MAJOR rustdoc from mandatory
pre-call requirement to recommended ABI-compatibility check; behaviour unchanged.
Finding #12: rewrite ad_get_tree rustdoc to direct observe-act agents to
ad_snapshot (refs, refmap, JSON envelope) and reserve ad_get_tree for ref-less
raw-tree consumers; removes stale "invoke the CLI" guidance.
* refactor: unify FFI command envelope serialisation and fix wait/status error contract
Extract write_command_envelope into commands/envelope_out.rs, eliminating
the ~35-line guard→serialize→string_to_c block duplicated across all five
command modules (snapshot, execute_by_ref, wait, status, version).
ad_wait and ad_status now write the error JSON envelope into *out on
command-level failures (TIMEOUT, ELEMENT_NOT_FOUND, etc.), matching the
behaviour already present in ad_snapshot and ad_execute_by_ref. Guard and
infrastructure rejections (null adapter/out/args, off-main-thread, invalid
UTF-8) continue to leave *out null — the infra/command boundary is
preserved exactly.
Add ad_wait_command_error_writes_error_envelope_into_out to verify the
unified contract: a zero-timeout element wait that cannot be satisfied must
produce an ok:false envelope in *out, not a null pointer.
* test: add unit tests to write_command_envelope proving #3 contract
* build: regenerate FFI header after round-1 review fixes
* docs(ffi): document ownership/lifetime contracts on adapter destroy and wait
Add Safety note to ad_adapter_destroy that callers must not destroy the
handle while any call on it is in flight on another thread — concurrent
destroy + in-flight call is use-after-free (confirmed reachable: destroy
is main-thread-exempt while ad_wait holds &*adapter through its blocking
wait::execute loop).
Add blocking-duration and adapter-lifetime note to ad_wait.
Extend ad_snapshot's existing partial note to cover explicit window
targeting (window_id, not yet ABI-exposed) alongside skeleton/drill-down,
and add CLI guidance for agents that need those features today.
* refactor(ffi): route ad_execute_by_ref through canonical core pipeline
Fix four coupled architectural findings in the FFI execute_by_ref path:
#2 (P1): Delete bespoke run_ref_action that bypassed CommandContext tracing
and hard-coded ref_id "<ffi>". Route ad_execute_by_ref through a new
commands::execute_by_ref::execute in core, which calls the same
execute_ref_action_with_context pipeline the CLI click/type/etc. commands
use — full trace, strict resolution, and actionability preflight.
#7 (P2): Add nullable snapshot_id: *const c_char parameter as the third
argument. Tri-state: null → latest snapshot (prior behaviour), valid UTF-8
→ pin that snapshot id, non-null invalid UTF-8 → ErrInvalidArgs. Passes
snapshot_str.as_deref() to RefArgs.snapshot_id mirroring CLI --snapshot
semantics. Update all 7 call sites in c_abi_lifecycle.rs and the extern
decl in tests/common/mod.rs (c_abi_actions.rs had no sites).
#10 (P2): Make core the single source of policy truth. Add
Action::base_interaction_policy() (delegates to may_use_focus_fallback —
TypeText + PressKey → focus_fallback, everything else → headless) and
InteractionPolicy::join(self, other) (bitwise-OR on the two capability
flags: elevate-only, never downgrade). FFI maps AdPolicyKind → core
InteractionPolicy and calls base.join(caller_ip); core::execute_by_ref
does the base+elevation. Delete the bespoke effective_action_policy fn and
its six duplicated unit tests from the FFI; repointed truth-table coverage
lives in the six FFI tests that delegate to core and the join unit tests in
interaction_policy.rs. Note: PressKey's base shifts headless→focus_fallback
vs the old FFI-only fn — intentional alignment with the full CLI table.
#6 (P1, light): Success and error paths already route through the shared
write_command_envelope helper (no reintroduction of bespoke serialization).
Add a /// note on ad_execute_by_ref documenting dispatch-before-serialize
ordering so callers understand the near-impossible ErrInternal-after-action
scenario without requiring heavy pre-validation machinery.
* build: regenerate FFI header after core-routing fixes
* test(ffi): split c_abi_lifecycle and add observe→act roundtrip tests
Finding #4: c_abi_lifecycle.rs was 1127 LOC, violating the 400-LOC cap.
Extracted focused modules each well under 400 lines:
- c_abi_init.rs (54 LOC) — ad_abi_version / ad_init
- c_abi_session.rs (88 LOC) — session adapter ctor
- c_abi_log_callback.rs(182 LOC) — log callback tests + statics
- c_abi_json_commands.rs(165 LOC) — ad_version / ad_status
- c_abi_snapshot.rs (131 LOC) — ad_snapshot guard + envelope tests
- c_abi_wait.rs (178 LOC) — ad_wait guard + error-envelope tests
- c_abi_execute_by_ref.rs(167 LOC) — ad_execute_by_ref guard tests
c_abi_lifecycle.rs shrunk to 176 LOC (null-safety + list lifecycle).
Finding #5: add snapshot→execute_by_ref roundtrip tests.
- stale_ref_returns_ok_false_error_envelope: always runs in CI; sets a
temp HOME (empty refmap), calls ad_execute_by_ref(@e1), asserts
ok:false + error.code when the command path executes, tolerates
ErrInternal/null-out for the macOS main-thread guard.
- snapshot_execute_by_ref_live_roundtrip: #[ignore]; needs AX permission,
a live app, and main-thread execution (E2E harness).
* fix(ffi): zero ad_wait *out before args guard; correct PressKey policy docs
Review of the review fixes:
- ad_wait now zeroes *out before the args null-check so a null-args
rejection honours the documented *out-zeroed contract (matches the
other command entrypoints).
- Corrected the PressKey policy doc wording: focus_fallback is the
shared base from Action::base_interaction_policy, not a claim of CLI
ref-PressKey parity (no CLI ref-PressKey action exists).
* chore: remove stray rust_out build artifact and gitignore it
* fix(ffi): thread session context through legacy ref-action path and correct doc steering
Add execute_entry_with_context to core so the FFI legacy path
(ad_execute_ref_action_with_policy) threads the adapter's real
CommandContext — built from the session id set at adapter creation —
instead of CommandContext::default(). The existing execute_entry
delegates to it with default() so all other callers are unchanged.
Rewrite the /// docs on all three native-handle / legacy-struct
entrypoints (ad_execute_action, ad_execute_action_with_policy,
ad_execute_ref_action_with_policy) to remove the false "CLI parity"
steering and describe each accurately as a low-level escape hatch with
verbatim-policy dispatch. Point observe→act callers at ad_execute_by_ref.
Add core tests confirming execute_entry_with_context succeeds, delegates
correctly, and emits trace events when a trace path is configured. Add FFI
test confirming the session-adapter path resolves to the same error class
as the no-session path (session id is wired into trace, not the error
surface).
* refactor: unify CLI ref-action policy via CommandContext::request_base
All 16 ref-action commands previously hardcoded InteractionPolicy literals
that duplicated the intent already encoded in Action::base_interaction_policy().
Add CommandContext::request_base(action) which reads the canonical base from
the action itself and delegates to request(), then replace every hardcoded
call site so there is a single policy source of truth for CLI and FFI.
Update the all_context_request_callers_are_policy_tested guard to search for
context.request_base( rather than context.request(.
Behavior is byte-identical: headless() and focus_fallback() literals were
exact mirrors of what base_interaction_policy() returns for each action.
* docs(ffi): fix Safety doc, add header preamble, document live-test deferral
Finding #9: correct the # Safety section on ad_execute_by_ref to distinguish
ref_id (non-null required; null is defined but rejected with ErrInvalidArgs)
from snapshot_id (null is meaningful — latest snapshot). The param tri-state
doc was already accurate; only the Safety section implied false symmetry.
Finding #10: add an agent-workflow orientation preamble to cbindgen.toml via
the after_includes key. The 13-line C block-comment lands after sys-includes
and before declarations (inside the include guard), covering the full
ad_init → ad_adapter_create → ad_snapshot → ad_execute_by_ref →
ad_free_string / ad_adapter_destroy loop with the macOS main-thread
requirement. Verified via local cbindgen 0.29.4 regeneration; header not
committed (orchestrator regenerates).
Findings #2/#8: expand module doc and live-roundtrip #[ignore] test doc to
precisely explain why the full observe→act loop cannot run under libtest on
macOS (off-main-thread scheduler + AX guard), how to run manually, and that
the full-loop CI proof is deferred to plan unit U9 / Phase B (Python ctypes
external-consumer harness). Strengthen the always-running error-envelope test
with an error.message assertion (guaranteed by the error contract) without
pinning error.code (which varies by load path; pinning would cause flakiness).
* build: regenerate FFI header after round-2 review fixes
* fix: correct FFI doc/test gaps found in PR #67 audit
#7: replace non-existent ad_click/ad_type_text references in ad_get_tree
rustdoc with the real entrypoint (ad_execute_by_ref + AdAction).
#6: extend cbindgen.toml preamble with snapshot_id round-trip (pass
data.snapshot_id back to pin snapshot vs NULL for latest), AdAction
construction (zero-init + set kind + kind-specific fields), and policy
semantics (0=keeps built-in base, 2=Headed allows cursor/focus fallbacks).
#2: add dispatch-before-serialize note to ad_snapshot matching the
existing note on ad_execute_by_ref — refmap is written before JSON
serialisation, so a serialisation failure leaves *out null + ErrInternal
while the refmap is already on disk.
#10: add out-of-range policy test in c_abi_execute_by_ref.rs that passes
policy=99 and asserts ErrInvalidArgs|ErrInternal + out null, with the
last-error assertion gated on ErrInvalidArgs to tolerate macOS off-main-
thread CI returning ErrInternal before the policy check is reached.
* fix: make error suggestions transport-neutral and populate retry_command
Finding #4: STALE_REF, AMBIGUOUS_TARGET, ELEMENT_NOT_FOUND, SNAPSHOT_NOT_FOUND,
NOTIFICATION_NOT_FOUND, and POLICY_DENIED suggestion strings previously referenced
CLI-only flags (--skeleton, --snapshot, --session, --headed) or bare CLI subcommand
syntax (snapshot, list-notifications), making them meaningless to FFI consumers.
Rewritten to name the operation neutrally with both CLI and FFI equivalents in
parentheses.
Finding #15: ErrorPayload::retry_command was always None; the with_retry builder
existed but was never called from from_app_error. Added retry_token_for_code() to
populate the field for mechanically retryable codes: STALE_REF and SNAPSHOT_NOT_FOUND
get "snapshot;execute_by_ref" (re-snapshot then re-execute by ref); POLICY_DENIED gets
"escalate_policy" (policy escalation, not a re-run of the same call). All other codes
remain None to avoid misleading auto-retry loops.
Tests updated: stale_ref_suggestion_mentions_skeleton renamed and tightened to assert
transport-neutral content; snapshot_ref_tests.rs skeleton assertion updated to match;
four new output.rs tests cover the populated tokens and the None cases.
* fix(trace): wire session_id into every JSONL record, improve FFI ref label, drop duplicate test
#9: thread session_id (Option<&str>) through TraceConfig::emit/emit_lazy/write_event and
insert it as a top-level field after the sanitized fields block. This makes the docs true
for everyone (CLI + FFI when both --session and --trace are provided). session_id is not
in SENSITIVE_KEYS and is intentionally inserted after sanitize_trace_value so it cannot
be shadowed or redacted. Omitted entirely when context has no session (no null key).
#8: replace the opaque "<ffi>" ref label in execute_entry_with_context with a
role/path-index-derived label (e.g. "<button/2/0/3>"). Uses only role and numeric path
indices — no content fields — so the label is safe to emit in the unredacted "ref"
trace key. Improves per-record correlation in multi-element FFI trace logs.
#14: delete execute_entry_delegates_to_entry_with_context from ref_action_tests.rs.
The test was byte-identical in assertions to failed_action_still_releases_resolved_handle
and added no new coverage. Removed rather than forced a weak rewrite.
Tests added/updated:
- context.rs: trace_injects_session_id_as_top_level_unredacted_field (session set),
trace_writes_jsonl_without_stdout_dependency extended with session_id absence assert
- ref_action_tests.rs: execute_entry_with_context_emits_trace_events extended to assert
session_id in every emitted record; trace_records_omit_session_id_when_context_has_none;
ref_label_from_entry_uses_role_and_path_indices
* fix(core): make stale-ref actionability suggestion transport-neutral; regen header
Mirrors the round-3 transport-neutral suggestion rewrite for the
live-staleness path in actionability, plus the regenerated FFI header
carrying the round-3 doc/preamble updates.
* refactor: dedup ffi policy mapping and error conversion helpers
Replace the duplicated AdPolicyKind→InteractionPolicy three-arm match in
execute_by_ref.rs with a canonical to_interaction_policy() method on
AdPolicyKind. Delegate action_request() in actions/execute.rs to the same
method, replacing its own redundant match. Replace the inline AppError→
AdapterError match in ad_execute_ref_action_with_policy with the shared
app_error_to_adapter helper already used everywhere else.
* refactor(ffi): apply ponytail/simplify cuts (then_some, trim core doc)
Use bool::then_some for ad_wait optional scalars; drop the FFI-history
sentence from the core execute_by_ref doc (the FFI doc retains it for C
binding authors). Behavior unchanged.
* refactor!: drop vestigial version --json flag and dead code
The tool emits a JSON envelope for every command and has no plain-text
mode, so the version --json flag toggled nothing. Remove it end-to-end:
CLI flag, core VersionArgs, batch path, dispatch/main call sites, and the
snapshot test. Correct the stale skill docs (they documented a non-existent
plain-string default and wrong platform/arch keys; actual keys are
version/target/os, always inside the JSON envelope).
Also: delete dead code (AppContext, WindowContext, Response::with_app and
their re-export; core BatchArgs); inline single-caller helpers
(strip_ref_bounds_when_hidden, helpers::find_window_for_pid,
permissions::render); use libc::kill instead of a hand-declared extern in
refs_lock.
BREAKING CHANGE: the version command no longer accepts --json; it always
emits the standard JSON envelope.
* refactor: inline single-caller ffi conversions, drop dead from_c
Inline direction_from_c and snapshot_surface_to_core into their sole
callers. Remove AdImageFormat::from_c (production converts Rust->C only;
the C->Rust direction was dead, #[allow(dead_code)]) together with its
unit test and fuzz line. Drop the ErrorCode/AdResult cardinality assert:
the exhaustive error_code_to_result match plus the per-discriminant pins
already enforce the ABI invariant.
* refactor: dedup keycode table, inline single-caller macos helpers
key_to_keycode delegated the same 52-entry table that
keyboard_map::key_name_to_code already owns; delegate to it, guarding the
modifier names so a modifier passed as the primary key still resolves to
None (preserving existing behavior). Inline windows_from_records,
is_window_candidate, make_output, and the normalized_role_for_element
wrapper (which let one caller classify the same element twice); collapse
accessibility_report_state to its one-line form. Swap four local
HashSet dedup sets to FxHashSet (rustc-hash, already a dependency).
* chore: mark internal crates publish = false
These crates are workspace-internal (consumed only via path deps; CI never runs cargo publish). Marking them unpublishable prevents accidental crates.io release and makes removing unreferenced pub items unambiguously safe.
* fix: finish version --json removal in ci smoke test and help text
The version --json flag was dropped earlier but two references survived:
the npm smoke-test in ci.yml (which now errored on the unknown flag and
failed the job) and the --help text that still advertised it. Point both
at plain version, which emits the same JSON envelope. Add a batch
regression test asserting version now rejects args, and drop the
misleading {"json":true} from the stop-on-error fixture.
* test: guard ErrorCode<->AdResult bijection after counter removal
Code review flagged that dropping the cardinality counters left reverse
drift (a new AdResult error variant without a matching ErrorCode)
unguarded. The counters only weakly approximated this anyway: they
compared two hand-maintained variant arrays to each other, decoupled
from the real enums.
Replace them with a stronger, leaner guard: an exhaustive reverse match
(error_code_origin) that fails to compile if an AdResult error variant
is added without declaring its ErrorCode origin, plus a round-trip test
asserting the forward and reverse maps agree -- a consistency check the
counters never performed. Document the bijection contract on
error_code_to_result.
