docs: update reliability docs and skills

This commit is contained in:
Lahfir 2026-06-02 02:06:11 -07:00
parent db25b4a030
commit 052ced9de3
5 changed files with 19 additions and 7 deletions

View file

@ -206,8 +206,8 @@ agent-desktop key-up shift # release key
```bash
agent-desktop hover @e3 # move cursor to element
agent-desktop hover --xy 500,300 # move cursor to coordinates
agent-desktop drag @e3 --to @e8 # drag between elements
agent-desktop drag --xy 100,200 --to-xy 400,200 # drag between coordinates
agent-desktop drag --from @e3 --to @e8 # drag between elements
agent-desktop drag --from-xy 100,200 --to-xy 400,200 # drag between coordinates
agent-desktop mouse-click --xy 500,300 # click at coordinates
agent-desktop mouse-down --xy 500,300 # press at coordinates
agent-desktop mouse-up --xy 500,300 # release at coordinates

View file

@ -36,8 +36,9 @@ Ref actions must pass through the shared reliability path:
2. Resolve the ref with strict platform identity checks.
3. Return `STALE_REF` when the old element no longer matches.
4. Return `AMBIGUOUS_TARGET` when multiple candidates match.
5. Run actionability checks before adapter dispatch.
6. Emit trace events only to the requested JSONL trace path, never stdout.
5. Run live actionability checks before adapter dispatch.
6. Keep waits bounded by the caller timeout, including platform resolution retries.
7. Emit trace events only to the requested JSONL trace path, never stdout.
## Cross-Platform Rule
@ -47,6 +48,12 @@ identity fields into the same `RefEntry` concepts: role, name, value,
description, state, bounds, supported actions, source surface, root ref, and
tree path.
Actionability should prefer one native live-state read that returns state,
bounds, and supported actions together. Platform adapters may fall back to
separate reads, but the CLI behavior must remain identical: empty transient
action reads do not erase snapshot capabilities, while a non-empty live action
set that lacks the required action can block dispatch.
## Review Rule
Any change to ref resolution or action dispatch must include tests for:
@ -54,6 +61,7 @@ Any change to ref resolution or action dispatch must include tests for:
- stale ref rejection
- ambiguous target rejection
- actionability failure before dispatch
- retrying waits that honor timeout and report last observed state
- session isolation
- FFI parity when the behavior is exposed through C ABI

View file

@ -92,7 +92,7 @@ Use **progressive skeleton traversal** as the default approach. It reduces token
- After any action that changes UI, re-drill the affected region or re-snapshot
- **Scoped invalidation:** re-drilling `--root @e3` only replaces refs from @e3's previous drill — refs from other regions and the skeleton itself are preserved
- **Strict resolution:** stale refs return `STALE_REF`; duplicate plausible targets return `AMBIGUOUS_TARGET` instead of choosing arbitrarily.
- **Actionability:** ref actions check visibility, enabled state, supported action, policy, and editability before dispatch.
- **Actionability:** ref actions check live visibility, enabled state, supported action, policy, and editability before dispatch.
- **Sessions:** use `--session <id>` for concurrent or multi-agent runs; batch entries may override with `"session": "id"`.
- **Trace:** use `--trace <path>` for JSONL diagnostics outside stdout; add `--trace-strict` only when trace write failures should fail the command.
@ -212,6 +212,7 @@ agent-desktop clipboard-clear # Clear clipboard
agent-desktop wait 1000 # Pause 1 second
agent-desktop wait --element @e5 --snapshot <snapshot_id> --timeout 5000 # Wait for element
agent-desktop wait --element @e5 --predicate actionable --timeout 5000 # Wait until actionable
agent-desktop wait --element @e5 --predicate value --value "Done" --timeout 5000 # Wait for value
agent-desktop wait --window "Title" # Wait for window
agent-desktop wait --text "Done" --app "App" # Wait for text
agent-desktop wait --menu --app "App" # Wait for menu surface

View file

@ -169,8 +169,11 @@ Pauses for N milliseconds. Use between actions that need time to settle.
### wait (element)
```bash
agent-desktop wait --element @e5 --snapshot <snapshot_id> --timeout 5000 --app "App"
agent-desktop wait --element @e5 --predicate actionable --timeout 5000
agent-desktop wait --element @e5 --predicate value --value "Done" --timeout 5000
```
Blocks until the element ref appears in the accessibility tree. Useful after triggering UI changes.
When `--snapshot` is omitted, the command polls the caller's latest session refmap and refreshes it on the built-in debounce. When `--snapshot` is passed, it stays pinned to that refmap. Element resolution is capped by the remaining `--timeout`, and timeout errors include the last observed predicate/actionability state.
### wait (window)
```bash

View file

@ -12,8 +12,8 @@ expectations.
| Snapshot refs | Refs are depth-first, snapshot-scoped, and persisted in the caller session |
| Strict resolve | A ref resolves only when identity still matches; stale refs return `STALE_REF` |
| Ambiguity | Multiple plausible matches return `AMBIGUOUS_TARGET`, never an arbitrary click |
| Actionability | Ref actions check visibility, enabled state, supported action, policy, and editability before dispatch |
| Wait recovery | `wait --element` can poll the latest session refmap when no snapshot is pinned |
| Actionability | Ref actions check live visibility, enabled state, supported action, policy, and editability before dispatch |
| Wait recovery | `wait --element` can poll the latest session refmap when no snapshot is pinned, honors the caller timeout while resolving, and reports the last observed predicate state |
| Session scope | `--session <id>` and batch item `session` never read or write another session's refmap |
| Trace | `--trace <path>` writes JSONL diagnostics outside stdout and is best-effort unless strict |
| FFI parity | FFI ref actions use strict resolve and actionability checks before adapter dispatch |