Verifier follow-ups: state plainly that AX_MESSAGING_TIMEOUT never became
an ErrorCode variant and AUTOMATION_PERMISSION_DENIED folded into
PERM_DENIED, and correct the ffi build.rs line count. Release-tracker
dates were re-verified against UTC tag creation dates and left unchanged
- the table is uniformly UTC, matching the GitHub release surface.
Bring docs/phases.md up to post-foundation reality and split the platform
phases into reviewable increments. Adds the completed Phase 1.6 section
(Playwright-grade contract, U0-U19) and a Platform Delivery Model: each
platform ships as dependency-ordered sub-phases (2.0-2.14 Windows,
3.0-3.14 Linux), one PR per sub-phase capped at 2,000 changed lines,
merging into a per-platform integration branch with plan->implement
lifecycle (no brainstorm stage) and a full-branch review before main.
Corrects stale content throughout: command roster 58 (54 operational +
4 fail-closed), capability-trait adapter listing, APP_UNRESPONSIVE and
envelope 2.1 recovery/disposition, FFI ABI major 3 with the shipped
handshake, hand-written FFI wrappers (codegen remains open P2-O16),
release tracker through v0.4.7, wait --event vs future watch naming,
nonexistent PLATFORM_UNSUPPORTED code, and the CLAUDE.md phase-number
mismatch. All prior research (capability maps, API mappings, rejection
tables, notification/tray approaches, risk register) is preserved and
folded into the owning sub-phases; compressed test-methodology bullets
restored into exit criteria after adversarial verification.
Document the head-vs-main comparison harness in the common commands and
close the agent guide with a gate: substantive changes end with
scripts/perf-baseline-compare.sh and a review of report.html against the
merge-base, so latency deltas are intentional rather than discovered by
users.
One command now measures the branch against its merge-base: builds both
revisions, runs a fixture A/B (isolated per-binary stores, alternating
rounds), optional strictly-read-only probes of real apps, and the
synthetic locator benchmark, then renders a self-contained report.html
with validated light/dark palettes, grouped p50/p95 bars, honesty
callouts for non-comparable defaults, and table fallbacks. Replaces the
markdown summary and the standalone locator-report generator, whose
sanitized-report flow required manual invocation and hardcoded sizes.
The FFI live driver addresses its AE2 targets by native_id per the
R4 identity spine, but the fixture's async buttons only ever set
accessibility labels, so ref_for could never match and both release-FFI
AE2 checks failed on every live run. These checks were unreachable
until the zero-bounds abort was fixed, which is why no prior run
surfaced it. Assign identifiers to exactly the three buttons the FFI
consumer addresses; name-based CLI paths are unaffected.
The live e2e gate exposed that find could never address the fixture's
zero-bounds synthetic element: AppKit answers kAXErrorFailure for
accessibilityActionNames on NSAccessibilityElements with no action
support and kAXErrorAttributeUnsupported for unimplemented settable
probes, and the action reader classified both as incomplete transport
reads, so selected-anchor hydration retried to TIMEOUT with
locator_selected_evidence_incomplete. Mirror the renderer-probe
absence classification for action reads: unsupported/no-value/
not-implemented/action-unsupported/failure are complete none-present
answers; CannotComplete, InvalidUIElement, and APIDisabled still mark
the read incomplete.
Resolve 15 validated findings from the 35-reviewer final pass: release the
interaction lease before read-only --wait-for polling; degrade transient AX
child-label errors instead of aborting whole snapshots; report honest
delivery evidence on chain exhaustion and disclosure verification; thread
drag delivery counts by reference; relax user-output overwrite validation
to symlink/dir refusal with INVALID_ARGS; sweep sessionless clipboard
images; correct false fail-closed FFI docs and state the deferred FFI
event-wait; unit-test the electron benchmark classifier; restore the
7-field/5-param standards caps via SurfaceWait, DragEndpoint,
WindowIdentityEvidence, NcSessionOps, ClickSpec, and PointResolveAttempt;
delete the dead non-CAS RefStore::save_existing_snapshot in favor of the
production CAS path.
Run strict headless click and type against visible background controls, validate only real fixture observables, and emit exact invariant diagnostics without changing fixture behavior.
Keep semantic ref actions strictly headless, make natural headed actions physical-first, and prevent notification observation from opening system UI without headed permission. Gate the native suite on background non-interference and assert delivered mechanisms.
Three failures that tonight's --no-verify checkpoints hid:
- nc_session.rs::nc_pid used Deadline::after (Result<Deadline>) where
operation_deadline yields Result<Instant> — E0308 broke Native check
(macOS/Windows) and the FFI cdylib builds. Reverted to the compiling form;
the speculative deadline floor did not fix NT3 anyway.
- resolve_query_tests.rs hit 403 lines; split the hydration adapters/tests
into resolve_query_hydration_tests.rs (both now well under 400).
- test_harness_contract.py pinned 9 abort_suite sites; the branch legitimately
has 10 (all required-fixture guards). Updated the invariant.
cargo check --all-targets --locked, clippy, fmt, source rules, harness
contract, and 1562 unit tests all green locally.
Remove the scroll-to preamble (the compacted fixture makes scroll-area
directly actionable; raw scroll verified 0->805). Assert disclosure state
by revealed child content instead of AXValue, which SwiftUI DisclosureGroup
does not expose — the product's expand_verified was truthful all along.
Product unchanged since 51c87b2 (1562 unit tests, clippy, release green);
these are test-harness shell edits only.
Containers no longer derive names from descendants (INTERACTIVE_ROLES gate)
and a per-node child-label cap no longer fails whole queries closed — the
root cause of every intermittent e2e abort. Fixture NSWindow double-release
(isReleasedWhenClosed) fixed: zero new crashes. Harness now pins the fixture
window to a deterministic frame on the largest display, clearing the whole
physical-gesture failure family. Selector-timeout diagnostic snapshot budget
raised 75ms->600ms so evidence capture can actually complete. Suite: 73 pass,
7 known remainders (scroll oracle, AE4 assert, three reliability envelope
extractions, sheet confirm lookup, NT1 observation).
Two root-cause fixes surfaced by a real session trace where clicking a
non-clickable group hung the full 5s auto-wait budget and returned an opaque
TIMEOUT whose failing check was unreadable.
Issue 1 (observability): ActionabilityCheck's check-identifier field was named
'name', colliding with the sensitive 'name' key the trace sanitizer redacts
(element accessible names, incl. Occluder.name). So the bounded-vocabulary
check identifier (visible/supported_action/...) was scrubbed from traces. Root
fix: rename the field to 'check' — a token the sanitizer leaves readable —
disambiguating it from Occluder.name, which stays correctly redacted. The
sanitizer keeps failing closed; the non-sensitive field just no longer collides.
Issue 2 (behavior): check_with_stability collapsed every actionability failure
to ACTION_FAILED, so the poll loop retried structurally-permanent failures for
the whole budget then returned a generic TIMEOUT. Root fix: model the intrinsic
transient-vs-terminal nature of each check. The terminal checks (supported_action,
policy, editable) — which waiting cannot heal — now carry their semantic permanent
code (ACTION_NOT_SUPPORTED / POLICY_DENIED); the report derives the error code
from the failing checks. The auto-wait poll loop's existing is_permanent_error
then fails them fast with a precise code, no poll-loop change. Transient checks
(visible/stable/enabled/receives_events) still surface ACTION_FAILED and retry.
Verified on a real app: click of a non-clickable scrollarea now returns
ACTION_NOT_SUPPORTED in 0.09s (was ~5s TIMEOUT) with the trace showing
{check: supported_action, reason: 'Click is not available'}. Full workspace tests
pass, e2e 72/0.
Follow-up polish on the actionability_timeout diagnostics (2nd-pass review P2s):
- Both failure sources now emit a phase-tagged envelope: resolve failures
{"phase":"resolve", code, message, details}, dispatch failures
{"phase":"dispatch", report}. Previously resolve emitted a resolve_error
wrapper while dispatch emitted a raw ActionabilityReport, so a consumer reading
the timeout report couldn't assume one shape.
- The dispatch branch now only overwrites last_report when it actually has
details; a details-less dispatch failure no longer wipes a prior resolve
failure's context. Verified with cargo test --workspace (1186 passed).
- actionability::visibility_check: run HIDDEN/OFFSCREEN state checks BEFORE the
bounds read (P1 #2). Previously bounds=None short-circuited to unknown, so a
hidden/offscreen element whose live bounds read failed slipped the visibility
gate. Regression tests added (hidden/offscreen + bounds=None now fail, not unknown).
- mouse_wheel: add unit tests (P0 #1) mirroring the sibling mouse-command tests —
args reach the adapter unchanged, the scrolled envelope shape, error propagation.
- ref_action_wait poll loop: on a retryable resolve failure, record the resolve
error into last_report so an actionability_timeout on deadline expiry carries
the last STALE_REF/AMBIGUOUS_TARGET context instead of an empty report (P2 #7).
Already-fixed in prior passes (review base predates 640e8e8): StepMechanism serde
tests (#3), clipboard trait removal doc note (#10). Deferred/accepted: test-adapter
boilerplate (#5 -> issue #95), hit-test ancestor + scroll thrashing (#15/#16/#17,
intentional design tradeoffs). Verified with cargo test --workspace (1186 passed).
The CLI command-module contract test (every_core_command_module_is_registered_or_declared_helper)
enumerates commands/*.rs and requires each to be a registered command or a
declared helper. The stale_retry_test_support module added in the #8 dedup
(640e8e8) ends in _test_support (not _tests), so it was neither skipped nor
allow-listed and failed CI. Add it to NON_COMMAND_MODULES alongside the sibling
helpers_test_support/wait_test_support entries. Local gap: this is a binary-crate
test that 'cargo test --lib' skips; 'cargo test --workspace' now green.
Document that hover/drag skip the actionability battery (occlusion-only via
require_receives_events, immediate ACTION_FAILED) while dispatch actions run
check_live's full battery and poll to TIMEOUT — the divergence that produced an
inaccurate skill-doc this cycle. Prevention: validate agent-facing capability
docs against the real per-command code path, not the mental model.
Reconcile four docs/solutions learnings against current code (their core
guidance is unchanged; only cited symbols/signatures/counts drifted):
- playwright-grade-desktop-reliability: add the actionability_timeout trace
kind; fix the check_actionability_with_trace signature (bundled
ResolvedRefAction) and the ref-allocation example (allocate_refs folds in
bounds-hiding; strip_ref_bounds_when_hidden is gone)
- real-app-tests-are-the-platform-adapter-gate: attribute the accessible-name
reducer to tree/builder.rs::accessible_name with element.rs::resolve_element_name
as the thin wrapper (unified in adda4c9)
- macos-gesture-headless-capability: 54 -> 58 commands; double_click now returns
Vec<ActionStep> tagged SemanticApi/PhysicalSynthetic
- exhaustiveness-guards: context.request( -> context.request_base(
Triage + validate (the review's line numbers were bogus diff-offset
artifacts, so every finding was checked against the real code) then fix the
real ones. 4 findings were false-positives (#2 core-internal fn, #5
VisibilityEvidence not in the actionability gate, #15 supertrait forward-design,
#20 as_str duplication), 2 were deferred as disproportionate (see below).
Verified by an adversarial code-read review + full gate: fmt, clippy
-D warnings, 1006 lib + 7 ABI tests, core isolation, 1.6MB, e2e 72/0, FFI
header regenerated with matching offset asserts.
- scroll gate: maybe_scroll_into_view now reads live state/bounds
(fail-open to the snapshot entry on capability-less adapters) so an element
that scrolled off-screen after the snapshot is still scrolled into view
- auto-wait: is_permanent_error now treats ErrorCode::Internal as permanent so
an internal dispatch error fails fast instead of retrying the whole budget
- FFI header: document ad_execute_by_ref's 5000ms auto-wait default (and the
timeout=0 single-shot escape hatch); correct the stale envelope version
2.0 -> 2.1; add the 22 per-field offsetof static asserts for AdRefEntry
- skill docs: document --modifiers on the mouse commands, the receives_events
occlusion check + occluder detail, the implicit scroll-into-view preflight,
and --timeout-ms — and correct the actionability section to reflect that
hover/drag run only the receives_events check (not the full battery) and
fail fast with ACTION_FAILED rather than polling to TIMEOUT
- adapter: /// note that get_clipboard/set_clipboard were removed pre-1.0 for
the typed content methods (C ABI unaffected)
- tests: StepMechanism serde roundtrip, ActionRequest legacy-no-timeout_ms
deserialization, execute_by_ref unit tests; dedup the byte-identical
StaleThenOkAdapter retry-counter into stale_retry_test_support
Deferred (disproportionate blast radius, tracked as follow-up issues):
process-state PID-reuse start-time corroboration (needs plumbing through the
already-large RefEntry + snapshot + platform adapter); a stub_ops! macro +
84-site test-adapter retrofit (large mechanical churn on the deliberate U0
four-trait split).
Remediate the validated PR #93 review findings. Each was triaged and
validated against the real code (0 false-positives; 4 deferred as
design/maintainability follow-ups), fixed on disjoint file sets, then
confirmed by an adversarial code-read review and the full gate (fmt, clippy
-D warnings, 996 unit tests, core isolation, 1.6MB binary, FFI codegen
drift-stable, e2e 72/0). Trace changes were additionally verified on a real
app: a ref-action click now emits ref.resolve.start/entry/ok, and a stale ref
emits ref.resolve.start/error(STALE_REF) with no dangling start.
- ref actions: emit ref.resolve.start/entry/error via a shared load_ref_entry
owner, so element-mutating actions trace resolution identically to get/is
(previously they emitted no start/entry, then a dangling start once added)
- scroll: surface a ref.scroll_into_view.error trace instead of silently
swallowing a failed scroll_into_view (non-blocking; the actionability check
still gates the action)
- actionability: attach the actionability report to the success trace so an
Unknown receives_events verdict is visible instead of vanishing
- launch: stop interpolating the raw app identifier into validate_app_identifier's
error message; carry it in redacted details.app_name (trace-leak fix)
- process_state: preserve original error details when upgrading to APP_UNRESPONSIVE
- execute_by_ref / resolve_point_with_wait: fold 7-arg signatures into config
structs (ExecuteByRefArgs; reuse PointResolveArgs) under the 5-param limit
- skills/README: correct the stale "56 commands" to 58
- tests: ProcessState::Crashed serde + Crashed/AppNotFound enrichment coverage,
requires_scroll_into_view exhaustiveness guard, normalize_action_timeout_ms(0)
guard, validate_app_identifier accept/reject + leak-regression, value-based
legacy ActionStep JSON comparison; split the process-state test file to stay
under the 400 LOC limit
Remediate the validated findings from the whole-branch review. Each was
triaged, validated against the real code, fixed on a disjoint file set, then
confirmed by an adversarial code-read review and the full gate (fmt, clippy
-D warnings, 985 unit tests, core isolation, 1.6MB binary, e2e 72/0). The
ref.resolve trace restoration was additionally verified on a real app (Finder
hover under a traced session emits start/entry/ok again).
- signals: propagate app-enumeration failures in the signal baseline instead of
swallowing them with unwrap_or_default, mirroring list_windows_impl
- image: validate the PNG signature and IHDR chunk in one shared core parser
(parse_png_dimensions), dedup three hand-rolled copies, and reject non-PNG
--image input instead of tagging arbitrary bytes as a zero-dimensioned PNG
- roles: delete the dead Role enum and inline is_interactive_role as a direct
INTERACTIVE_ROLES membership check, keeping the vocabulary-contract test
- helpers: cap each ref-resolve attempt to the wait deadline through the single
traced resolver, so budgeted hover/drag can no longer spend the whole budget
on one slow attempt and once again emit ref.resolve.* trace events
- actionability: run the hit_test occlusion gate even when get_live_element is
unsupported, so occlusion is checked on hit_test-capable adapters
- launch: share the app-identifier traversal guard across both launch paths
- window_resolve: fail closed when the window-title fallback is ambiguous
(resolve only when exactly one window matches the title)
- main: surface stdout write/flush failures from the JSON envelope instead of
reporting success, exiting with a distinct code on a broken pipe
- cli_args: add a one-to-one Surface<->SnapshotSurface parity tripwire test
Addresses the code review's remaining validated findings (test + robustness
hardening; no P0 correctness left after the name and window-bridge fixes):
- find --window-id: MockAdapter test (two windows, distinct trees) proving the
flag scopes the search to the requested window and a swap with app would fail.
- WindowScope serde: batch-JSON back-compat + typo-rejection tests for the
flattened app/window_id group on find/snapshot/screenshot.
- Offscreen bounds: extracted window_bounds_for_children as a pure helper and
restored the AXWindow-own-bounds / inherited-fallback unit coverage that was
lost when the live query matcher was deleted.
- Real-app guards: the #[ignore] snapshot_test.rs guards now print a SKIP reason
before each early return, so a no-op run is visible instead of a silent pass.
- Hit-test occluder naming now applies promoted_label.or_else(resolve_element_name),
matching the strict resolver and ambiguity classifier.
- Hover e2e asserts a clean (non-"hovered") baseline after the cursor reset.
Verified: e2e 71/0, Finder sidebar cells still re-resolve, clippy + workspace
tests green.
The earlier window fix replaced the nonexistent AXWindowNumber attribute with
_AXUIElementGetWindow in window_resolve.rs only; code review found three more
live sites still reading AXWindowNumber (which AppKit/SwiftUI never publish, so
it always returns None):
- resolve_roots.rs window_by_number: scope_verified could never become true, so
strict ref re-resolution silently degraded to bounds-hash-only matching and
failed closed (STALE_REF) on any window move/resize.
- window_inventory.rs: the AX-fallback path stamped every window id "w-0"
(collisions), and focus detection fell back to title-equality — wrong for any
app with two or more same-titled windows.
Extracted ax_window_id (the _AXUIElementGetWindow bridge) as the single shared
owner and routed all three sites through it; removed the now-dead copy_i64_attr
helper entirely (its only callers were the AXWindowNumber reads); switched the
bridge's success check to kAXErrorSuccess; and added a verified-title fallback
so a transient _AXUIElementGetWindow error on an already CG-verified window
retries by title instead of returning a false WINDOW_NOT_FOUND.
Verified: list-windows focus detection returns non-zero ids, e2e 71/0.
Code review found the STALE_REF name-divergence class was re-introduced: the
snapshot builder stores a ref's name via its own chain (title -> description ->
static value -> label_from_children child text), but resolve_element_name — used
by strict ref re-resolution — dropped the child-label rung and trimmed blanks
differently. So an interactive element named only by descendant text (Finder /
Mail / System Settings sidebar cells) or by a whitespace/blank title stored one
name and recomputed another, failing identity_matches -> STALE_REF on
click/type/get. Confirmed: 5/5 Finder sidebar cells returned STALE_REF.
- One shared reducer `builder::accessible_name` (title -> description ->
static-text value -> aggregated child label, each trimmed and blank-as-absent),
with the own-text portion factored into the pure, unit-testable
`reduce_text_name`. Both the snapshot builder and resolve_element_name reduce
through it, so a stored ref name always equals what the resolver recomputes.
- Deleted the now-single-producer/single-consumer NameEvidence indirection
(crates/core/src/accname.rs, crates/macos/src/tree/name_evidence.rs) and the
now-dead label_from_child_attrs.
- Added reduce_text_name unit tests covering the rung precedence and the
blank/whitespace handling that accname_tests used to guard.
Verified: 5/5 Finder sidebar cells now re-resolve, e2e 71/0, clippy clean,
workspace tests green.
Records the durable learning from the foundation-branch remediation: green
mock/stub unit CI cannot cover platform-adapter mechanics (window bridge,
accessible-name computation), so the e2e and #[ignore] real-app tests are the
mandatory gate before merging adapter changes. Cross-references the existing
playwright-grade reliability contract.
