docs: refresh reliability/actionability solutions for symbol drift

Reconcile four docs/solutions learnings against current code (their core
guidance is unchanged; only cited symbols/signatures/counts drifted):
- playwright-grade-desktop-reliability: add the actionability_timeout trace
  kind; fix the check_actionability_with_trace signature (bundled
  ResolvedRefAction) and the ref-allocation example (allocate_refs folds in
  bounds-hiding; strip_ref_bounds_when_hidden is gone)
- real-app-tests-are-the-platform-adapter-gate: attribute the accessible-name
  reducer to tree/builder.rs::accessible_name with element.rs::resolve_element_name
  as the thin wrapper (unified in adda4c9)
- macos-gesture-headless-capability: 54 -> 58 commands; double_click now returns
  Vec<ActionStep> tagged SemanticApi/PhysicalSynthetic
- exhaustiveness-guards: context.request( -> context.request_base(
This commit is contained in:
Lahfir 2026-07-05 01:22:27 -07:00
parent 640e8e83e4
commit 604844d22d
4 changed files with 55 additions and 20 deletions

View file

@ -80,12 +80,12 @@ const POLICY_TESTED_COMMANDS: &[&str] = &[
#[test]
fn all_context_request_callers_are_policy_tested() {
// scans crates/core/src/commands/*.rs (excluding *_tests) for files
// containing `context.request(` and fails, naming each stem, when one
// containing `context.request_base(` and fails, naming each stem, when one
// is absent from POLICY_TESTED_COMMANDS
}
```
The universe is not hand-maintained: the test scans the filesystem for the call-site signature every ref-action command shares. A new command file that calls `context.request(` without a registered policy assertion fails CI with a message naming the stem and the required follow-up.
The universe is not hand-maintained: the test scans the filesystem for the call-site signature every ref-action command shares. A new command file that calls `context.request_base(` without a registered policy assertion fails CI with a message naming the stem and the required follow-up.
**Leg 3 — per-case value pins.**

View file

@ -53,7 +53,7 @@ The answer is **per-gesture and per-platform**, because a gesture is headless-ca
## Why This Matters
- It keeps the **headless-first reliability guarantee** honest: the tool only claims a headless effect when the OS actually provides one, and fails closed otherwise.
- It preserves **cross-platform extensibility**: the same 54-command surface works identically across macOS/Windows/Linux, and each adapter contributes whatever headless capability its platform has — without touching the command layer.
- It preserves **cross-platform extensibility**: the same 58-command surface works identically across macOS/Windows/Linux, and each adapter contributes whatever headless capability its platform has — without touching the command layer.
- It prevents the **vacuous-success trap**: assuming `ok:true` means the gesture happened, when an AX action succeeded at the API layer but the control ignored it.
## When to Apply
@ -79,11 +79,19 @@ The macOS dispatch gates the physical path on the policy (so it is reachable onl
```rust
// crates/macos/src/actions/chain_defs.rs
pub(crate) fn double_click(el, _caps, policy) -> Result<(), AdapterError> {
pub(crate) fn double_click(
el: &AXElement,
policy: InteractionPolicy,
) -> Result<Vec<ActionStep>, AdapterError> {
if ax_helpers::has_ax_action(el, "AXOpen") && ax_helpers::try_ax_action(el, "AXOpen") {
return Ok(()); // headless AX path
return Ok(vec![
ActionStep::succeeded("AXOpen").with_mechanism(StepMechanism::SemanticApi), // headless AX path
]);
}
crate::actions::dispatch::click_via_bounds(el, MouseButton::Left, 2, policy) // gated; POLICY_DENIED headless
crate::actions::dispatch::click_via_bounds(el, MouseButton::Left, 2, policy)?; // gated; POLICY_DENIED headless
Ok(vec![
ActionStep::succeeded("CGClick").with_mechanism(StepMechanism::PhysicalSynthetic),
])
}
```

View file

@ -134,8 +134,11 @@ The reliable split is:
details carry a `kind` discriminant: `"wait_timeout"` for wait-loop expiry
(predicate, timeout_ms, last observed state) and `"chain_deadline"` for a
chain step expiring mid-increment or mid-disclosure (observed value or
expanded state, plus a `mutated` flag) — agents key on `kind` before
inspecting other fields.
expanded state, plus a `mutated` flag), and `"actionability_timeout"` for
the ref-action auto-wait poll loop (click/type/set-value/etc.) exhausting
`--timeout-ms` while resolution or actionability keeps failing transiently
(carries an optional `report` field with the last actionability failure
detail) — agents key on `kind` before inspecting other fields.
- For `wait --element` without `--snapshot`, refresh the latest-ref cache on a
bounded cadence; for a fixed `--snapshot`, treat missing refs as invalid input
instead of silently switching snapshots.
@ -261,8 +264,20 @@ presentation bounds when needed:
```rust
let identity_opts = opts.with_ref_identity_bounds();
let tree = adapter.get_tree(&window, &identity_opts)?;
let (tree, refmap) = allocate_refs(tree, opts)?;
let tree = strip_ref_bounds_when_hidden(tree, opts);
let mut refmap = RefMap::new();
let config = RefAllocConfig {
include_bounds: opts.include_bounds,
interactive_only: opts.interactive_only,
compact: opts.compact,
pid: window.pid,
source_app: Some(window.app.as_str()),
source_window_id: Some(window.id.as_str()),
source_window_title: Some(window.title.as_str()),
source_surface: opts.surface,
root_ref_id: None,
path_prefix: &[],
};
let tree = ref_alloc::allocate_refs(tree, &mut refmap, &config);
```
Ref action execution should keep the command-selected policy while centralizing
@ -270,7 +285,8 @@ the strict ladder:
```rust
let (entry, handle) = resolve_ref_with_context(ref_id, snapshot_id, adapter, context)?;
check_actionability_with_trace(ref_id, &entry, handle.handle(), adapter, &request, context)?;
let target = ResolvedRefAction { adapter, entry: &entry, handle: handle.handle(), ref_id, context };
check_actionability_with_trace(&target, &request)?;
let result = adapter.execute_action(handle.handle(), request)?;
```

View file

@ -92,12 +92,16 @@ a test that drives the real platform. Invest there, not in a more elaborate mock
The `STALE_REF` regression was caused by three copies of "what is this element's
name" drifting apart. An element's accessible name must be computed by a single
canonical function (`crates/macos/src/tree/element.rs::resolve_element_name`)
that the snapshot builder, the live matcher, and the strict ref resolver all
call. A ref's stored name and the resolver's recomputed name must come from the
same code, or freshly-created refs go stale. Any new consumer of an element's
name or identity must call the canonical resolver — never re-read `AXTitle` (or
any single attribute) itself.
canonical reducer, `crates/macos/src/tree/builder.rs::accessible_name`
(precedence: title → description → static-text value → aggregated child
label), which the snapshot builder calls directly when it stores a ref's name.
`crates/macos/src/tree/element.rs::resolve_element_name` is a thin
AXElement-only wrapper (`accessible_name(el, &fetch_node_attrs(el))`) called by
the strict ref resolver, the live matcher, and hit-test occluder naming. A
ref's stored name and the resolver's recomputed name must come from the same
reducer, or freshly-created refs go stale. Any new consumer of an element's
name or identity must call the canonical reducer or its wrapper — never
re-read `AXTitle` (or any single attribute) itself.
## Why This Matters
@ -157,10 +161,17 @@ assert!(get(ref_id, property = "role")["ok"]);
And the single-owner name computation the resolver, matcher, and builder share:
```rust
// crates/macos/src/tree/element.rs — the one accessible-name owner.
// crates/macos/src/tree/builder.rs — the one accessible-name reducer.
pub(crate) fn accessible_name(el: &AXElement, attrs: &NodeAttrs) -> Option<String> {
// title, else description, else (static text only) value, else a label
// aggregated from descendant text.
}
// crates/macos/src/tree/element.rs — thin wrapper the strict resolver, live
// matcher, and hit-test occluder naming call; the snapshot builder calls
// accessible_name directly.
pub fn resolve_element_name(el: &AXElement) -> Option<String> {
// AXTitle, else AXDescription, else (static text only) AXValue.
// Every name/identity consumer calls this; none re-reads AXTitle itself.
accessible_name(el, &fetch_node_attrs(el))
}
```