mirror of
https://github.com/lahfir/agent-desktop.git
synced 2026-08-09 00:17:27 +00:00
42 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Lahfir
|
4f7bad337c
|
feat: windows input synthesis (sub-phase 2.8) (#123)
* docs: add sub-phase 2.8 input synthesis plan * docs: harden the 2.8 input synthesis plan after adversarial review * docs: give every 2.8 deferral a receiving owner in phases.md * feat: measure windows input synthesis gaps (area 20) Probe the unmeasured SendInput and token facts 2.8 depends on so coordinate, abort-drag, double-click, integrity, and cost branches are settled before the adapter InputOps land. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows mouse_event via SendInput Give bare-coordinate hover/click/wheel a real physical path with primary-vs-virtual-desktop normalization, FFI held-edge guards, and a release-safe modifier hold around multi-click. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows drag with origin release guard Port the macOS abort-state contract onto SendInput so an interrupted drag posts corrective release at the origin and reports delivered_unverified instead of claiming a clean failure. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows keyboard synthesis and blocked combos Ship chord and UTF-16 type_text primitives with release guards, macOS key-name parity, and a Windows-specific dangerous-combo list so press/type can land without stuck keys or silent session-kill shortcuts. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: detect uipi elevation before physical input Compare caller and target integrity levels so a Medium-into-High SendInput attempt fails closed as PERM_DENIED instead of reporting a silent no-op as success. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: wire windows physical execute_action legs Replace the 2.7 not-supported stubs for type, press, multi-click, and right-click with SendInput synthesis gated by focus, elevation, and headed policy. Co-authored-by: Cursor <cursoragent@cursor.com> * test: pin windows physical input envelope parity Lock AdapterError and ActionResult wire shapes for the physical legs against the macOS disposition contract, and cite the A20-6 hot-path cost baseline already captured in area 20. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: dogfood windows input synthesis and harden delivery Prove the physical path on Notepad and ScratchForms under the corpus safety envelope, and fix the dogfood-found root-window foreground gate plus SendInput absolute normalization and drag pickup timing. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: align windows input synthesis docs with shipped evidence Restate phases, concepts, skill wording, and ledger rows so 2.8 reads as what shipped, including UIPI detection closure and the type command divergence handed to 2.15. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: harden windows input abort guards after review Strip plan decision ids from crates so CI phase-reference scan passes, arm a click release guard around each down/up, and re-verify keyboard focus at every type_text chunk boundary. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor: give the mouse input constructors and release-guard state one home button_input existed verbatim in four files and move_input in three, so the corrective batch an abort depends on could drift from the normal post. The click and drag guards also carried the same armed-and-counted state machine and the same delivery report, differing only in their recovery hint. Both now have a single home: mouse_send.rs owns the SendInput surface - the MOUSEEVENTF_* flags with their compile-time assertions against windows-sys, plus the two input constructors - and release_state.rs owns the guard state and the delivery report it produces. Each guard keeps its own Drop, because only the guard knows whether the corrective post is a button-up in place or a move back to the origin first. * fix: resolve the host window before refusing a physical click target_window_is_foreground read the leaf element's NativeWindowHandle, which UIA reports as 0 - success, not failure - for every element that is not itself a window. That is the normal shape for WPF, WinUI, UWP and Chromium content, so double-click, triple-click and right-click were refused before injection on most modern desktop UI. The handle is now resolved the way the rest of the crate resolves it, by climbing to the first ancestor that owns one. Also from the same review pass: - mouse-wheel dropped the caller's modifiers, so the documented --modifiers shift horizontal-scroll chord silently degraded to a plain vertical scroll while still reporting success. - The drag preflight reserved one pickup delay where the sequence spends two, admitting drags that then aborted mid-gesture with the button down. Both delays now come from one constant. - The skill docs carried plan decision ids and sub-phase numbers, and they are include_str!d into the binary, so agent-desktop skills served them to agents. The phase-reference scan now covers skills/ for exactly that reason and caught a second instance in SKILL.md. * fix: close the remaining input-synthesis review findings - The click sequence counted delivered events per click and let the inter-click gap fail through a bare ?, so a --count 3 that aborted late reported two events, or none at all with an unknown disposition. Accounting now spans the sequence while cleanup stays per click. Wheel chunks report the same way: a partial scroll says how much landed. - mouse_modifier opened a second SendInput keyboard path with its own fake sink, so the one-seam invariant keyboard_send documents was already false. It now posts through that seam and reuses its VK map. - type_text's focus gate and per-chunk re-verify were exercised through a cfg(test) clone that omitted the policy gate, the preflight and the real verifier, so deleting either guard left the suite green. The gate is one production function now, with the verifier injected, and a test drives it with a verifier that fails mid-sequence. - Two drag tests asserted a std-library identity and re-typed the production step arithmetic. They drive synthesize_drag and read the posted event stream instead. Inverting the first showed the zero-dwell early return was dead - the loop guard already covered it - so it is gone. - elevation's module doc blamed unavailable token manufacture for the unproven cross-boundary effect; the captures record manufacture working with no higher-integrity target to aim at. keyboard's docs claimed its primitives were unreachable from execute_action, which calls both. * docs: capture the input-synthesis learnings and correct what the run overstated Two new solutions docs and one extended: - An enforcement gate must cover everything the binary embeds. The phase-reference scan read crates/ and src/ for .rs files while the skill markdown it never looked at was include_str!d into the binary, so plan ids were being served to agents by the skills command. Widening the scope immediately found a ninth instance nobody had reported. - A zero success value is not the answer you asked for. UIA returns NativeWindowHandle 0 as a success for any element that is not a window, and the crate already had the climb that treats it as keep-looking; the physical-click gate re-derived a naive version and refused delivery on WPF, WinUI and Chromium. - The cannot-fail taxonomy gains the cfg(test) re-implementation shape, and a prevention note to invert one production site per run: two mutations at once made a test fail on the other one's arithmetic and hid the failure that mattered. The dogfood report claimed the multi-click path more broadly than it measured it. Both multi-click judgements ran against HWND-bearing targets, which is precisely where the defect was invisible, so J4 now states its scope and the residual records that a non-HWND judgement is still owed. * docs: judge multi-click on a non-HWND target and close the residual The original run judged multi-click only against WinForms and Notepad, both of which own a window handle - the one shape where the foreground gate's leaf-handle read could not fail. J10 runs the same WPF button with the pre-fix and fixed binaries back to back: the first reports ACTION_FAILED and not_delivered with the app's click counter untouched, the second lands and the counter advances to action:2, read back independently of the envelope. Two correct refusals fell out of the same run and are recorded with it: the occlusion gate naming the terminal window that actually covered the fixture, and the private-file guard rejecting a store an elevated run had left owned by the Administrators group rather than the user. * fix: block modifier supersets of a dangerous shortcut Exact canonical-string matching let alt+shift+tab through to SendInput. That is the reverse task switcher and takes the foreground exactly as the alt+tab it extends, which is the harm the list exists to prevent. A test asserted the old behaviour as correct, reasoning that an extra modifier makes a shortcut unrelated; shift does not neutralize alt+f4 and does not neutralize alt+tab, so the premise was wrong and the test is gone. Matching is now key equality plus modifier superset. It over-blocks a few combinations nobody presses, and that trade is deliberate: a wrongly blocked combo costs one --force, a wrongly allowed one moves input to another window mid-run. macOS answers the same problem by enumerating variants (it lists cmd+q and cmd+shift+q separately), so the two adapters now decide danger by different rules. Recorded as a settlement item for the hardening gate rather than changed here, since the macOS crate is the GA line. --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Lahfir
|
5ed56337ac
|
feat: windows semantic action tier (sub-phase 2.7) (#122)
* docs: add sub-phase 2.7 semantic action tier plan * feat: measure windows semantic action write surface (A19) Probe area 19 establishes COM-stack viability for the 2.7 dispatch tier — pattern invoke, failure taxonomy, secure writes, SetFocus foreground, Legacy click, combobox/ladder geometry, and cost — so later units implement against measured branches rather than inference. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add windows mutation classifier and retrofit ScrollIntoView Write failures classify through one UiaFailure table with pinned code and disposition arms; ScrollIntoView keeps observation-owned delivery while overlays classifier code and platform detail. Scan governance covers every mutation file before new pattern call sites land. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: wire windows execute_action with click chain and honest arms Override ActionOps::execute_action so Click runs Invoke then Legacy DoDefaultAction (A19-6), SetFocus is headed-gated (A19-5), and every other Action variant returns a deliberate outcome instead of the trait default. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add windows value writes with secure-field withholding SetValue and Clear dispatch through Value and RangeValue chains; IsPassword tri-state gates every value re-read so secrets are never echoed and secure writes report verified none with delivered_unverified honesty. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add windows toggle and disclosure semantic actions Headless Toggle/Check/Uncheck and Expand/Collapse run through pattern chains with observed-state verification, tri-state double-toggle (A19-2), and LeafNode fall-through instead of blind Invoke. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add windows select and scroll semantic actions Select resolves names through SelectionItem self-match or bounded descendant search with expand-first combobox handling; Scroll drives ScrollPattern with per-axis honesty and percent or bounds verification. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: add windows ancestor scroll ladder for ScrollTo Non-ScrollItem targets scroll into view through ScrollPattern ancestors with geometry-directed increments and the 2.6 visibility predicate; unsupported terminals remain only when no scrollable ancestor exists. Co-authored-by: Cursor <cursoragent@cursor.com> * test: pin windows action envelope parity and cost baseline Serialize ActionResult and classifier dispositions against the macOS wire shape in CI, and keep the A19-8 min/median/max cost captures as the tier's first hot-path baseline. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: dogfood windows semantic action tier against live targets Drive the release binary through Notepad, Explorer, scratch fixtures, and Obsidian; judge by quoted envelopes so click no longer names execute_action and residuals land with owning scopes. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: align phases and skills with shipped windows semantic actions Correct execute_action naming, Legacy click and SetFocus policy, mutation classifier and secure-field vocabulary, and the platform actions folder map so the product docs match A19 and the 2.7 dogfood. Co-authored-by: Cursor <cursoragent@cursor.com> * style: rustfmt windows semantic action modules Co-authored-by: Cursor <cursoragent@cursor.com> * refactor: share windows action write helpers across modules Lift classify_write, verification window capping, and scroll/visibility predicates into shared helpers so per-action modules stay thin without changing delivery semantics. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: honor post-delivery disposition and ladder absence honestly Map toggle/disclosure observation failures after a real write through after_delivery, refuse to count ScrollPattern absence as ladder delivery, and split oversized dispatch tests under the 400-line cap. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: export ladder absence pin for unit coverage Co-authored-by: Cursor <cursoragent@cursor.com> * style: drop non-doc comments from envelope parity tests Co-authored-by: Cursor <cursoragent@cursor.com> * fix: stop silent wrong delivery in select, scroll, and toggle Duplicate SelectionItem names now return AMBIGUOUS_TARGET, ancestor scroll refuses invented Down directions, Check/Uncheck no longer Invoke-falls-through after an unverified Toggle delivery, and the UIPI probe uses New-NotMeasuredResult so CI cannot silent-pass an unmeasured leg. * fix: realize before select so virtualized duplicates stay ambiguous A first-pass SelectionItem hit no longer skips scroll-to-realize or short-circuits the second search, so a below-fold duplicate can still surface as AMBIGUOUS_TARGET before Select fires. * fix: keep select ambiguity and scroll disposition honest across steps Retain the first SelectionItem match across realize/re-search so a duplicate revealed only after scrolling is AMBIGUOUS_TARGET, and stamp later ladder scroll failures delivered_unverified once the viewport moved. * fix: search select matches after every realize scroll window Virtualized lists can expose same-name items only between scroll steps; searching solely at the end misses intermediate duplicates and keeps a silent first hit. --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Lahfir
|
85f3b6e76a
|
feat: windows actionability and occlusion (sub-phase 2.6) (#121)
* docs: plan the windows actionability and occlusion work * docs: close the critique round on the actionability plan * test: measure windows actionability unknowns for hit-test and scroll A18 probes pin ScrollIntoView observation arms, unclipped straddling geometry, hang-bounded ElementFromPoint, virtual-screen Unknown guards, and WPF zero-size envelope staging before product code relies on them. * feat: add windows hit_test with never-false-negative classification ElementFromPoint classification reaches ReachesTarget or Unknown under the closed Err set, with virtual-screen and minimized guards so phantom desktop hits cannot become false interceptions before corroboration. * feat: corroborate windows hit_test interception with window attribution InterceptedBy fires only on the agreed same-root or cross-window arms, with redaction-safe occluder evidence, so false negatives stay Unknown. * feat: add windows scroll_into_view judged by observation Gates ScrollItemAvailable and verifies delivery from re-reads so the auto-scroll retry reports an honest unsupported outcome instead of PLATFORM_NOT_SUPPORTED. * test: pin windows actionability envelopes for disabled zero-bounds occlusion Fake-driven checks keep macOS-parity codes and dispositions in CI, with live disabled evidence and an opt-in WPF zero-bounds leg. * test: dogfood windows actionability against foreign occluders Exercises headed occlusion and scroll seams on real targets, unblocks window resolve for the headed path, and records envelope-backed judgements. * docs: align phases and concepts with windows actionability evidence Restates the hit-test and scroll contract from A18/U6 measurements and adds the shared actionability vocabulary the adapter now implements. * docs: align phases and concepts with windows actionability evidence Restates the hit-test and scroll contract from A18/U6 measurements and adds the shared actionability vocabulary the adapter now implements. * style: format windows actionability modules and harden live hit skips Keeps rustfmt clean and soft-skips on-screen reachability when a foreign desktop occluder covers the fixture under Cursor. * fix: require viewport evidence for scroll verification Missing viewport no longer counts as verified visibility, and the UIA client is acquired before ScrollIntoView so post-invoke errors stay observation-judged. * fix: revalidate window ownership at the focus point of use * style: fold probe commentary into doc comments * docs: state the file-size cap's enforcement scope * fix: qualify headed focus success by live window ownership * fix: bound headed focus and window resolution by the operation deadline * docs: record the window-focus surface and hand its policy depth to 2.9 * docs: tell agents what the windows actionability failures mean * test: pin that focused only ever means owned foreground * refactor: split window resolution tests into a sibling module * test: execute the real scroll gate and ban the whole read table * fix: make dogfood judgements reject the defects they hunt * test: scan the occlusion gate for banned calls and name rendering * test: make the windows lane stage the live wpf evidence it claims * fix: budget the occlusion gate's walks and scope viewport demotion to same-root * docs: state the measured deadline overshoot for the occlusion gate * refactor: give the duplicated geometry and walker helpers one home * fix: stop a discarded viewport walk from unmaking a settled verdict * fix: make the live occlusion legs immune to desktop churn * docs: give off-screen fixture parking a measurement owner * test: dogfood the hardened occlusion gate and record the judgement rebuild * docs: fold the actionability lessons into the solutions corpus * fix: shrink the headed focus write surface to the foreground call * fix: guard headed focus with full process identity at every point of use |
||
|
Lahfir
|
adf2c36bde
|
feat: windows resolution and live locator (sub-phase 2.5) (#120)
* docs: plan the windows resolution and live locator work
Eight units grounded in the probe corpus, the shipped 2.4 resolver, the
macOS reference implementation, and core's live-locator contracts. The
research reframed the work honestly: 2.4 already shipped a binary
exact-evidence resolver, so 2.5 upgrades it to graded three-state matching
with the fingerprint fallback Electron's measured 0% AutomationId coverage
demands, implements resolve_locator_anchor so find stops failing, and
builds the five live readers over the machinery earlier sub-phases
hardened.
A five-persona review ran before commit and its findings are folded in:
the KTD3 composition rule (core's identity_match alone would undo the
A7-3 wrong-target defense - two core verdicts compose, with
stable_text_match as the sanctioned visibility promotion), mid-search
element-vanish routed incomplete-retryable rather than terminally stale,
the reader-path liveness corroboration given a real data path through the
element wrapper, the Electron path-survival measurement added before the
tier that relies on it, the path fast-path eligibility gate and the
positive-area geometry gate, shape-only ambiguity details kept against the
macOS pattern that would violate the redaction gate, and the secure-field
question split read-side/action-side with the action remainder routed to
its owner.
* docs: close the second review round on the resolution plan
The round hunted second-order defects in the first round's fixes and found
real ones. The retryability stamp is one write, not two - a details bool
that IS the typed channel - so the completed-search stale ref now mirrors
the macOS complete-absence shape instead of a blanket non-retryable that
would have disabled core's hydration retry and failed find permanently on
first churn. The anchor variant settles path misses immediately rather
than replaying a stale path inside the adapter loop, so core's fresh
re-observation - the actual cure - keeps the deadline. The KTD3
composition gains the no-text-identity gate that kept icon-only refs from
spinning retryable, keyed on the live field rather than the verdict. The
degenerate-hash ref class is named unresolvable-by-design with the census
attributing it, the handle-payload change names its downcast consumers,
the KTD10 write-back joins U8's enumerated list, the Electron probe leg
gets its repo-controlled vault and unmeasurable branch, and four
edit-introduced contradictions from round one are reconciled.
* docs: align the resolution plan's done criteria with the second review round's decisions
* feat: measure the windows resolution unknowns (probes 17-resolution)
Stand up the A17 probe family for sub-phase 2.5 so no resolution or live-read
fact ships unmeasured: find_all versus the shipped walk on both fixtures, the
live 0/1/N candidate counts over duplicate-evidence pairs, the single-element
strict-resolve and shared-read timing envelopes, and the ambiguity census over
the WPF and Win32 trees. The WinForms fixture gains an equal-count ListBox
swap and a duplicate-evidence button pair; the swap arm triggers the mutation
by WM_APP because the custom AutomationId provider suppresses Invoke. The
Electron path/geometry leg stages a repo-controlled Obsidian vault; the first-
contact shell and an unstageable marker tree are recorded per row with U7's
judged rate as the measurement of record. A17 rows added to the ledger with
the pre-committed branches taken, the hunk index re-synced to the measured
diff, and the probe wired into the capability-probe workflow for the second
environment.
* feat: make windows resolution matching three-state
Replace the boolean candidate comparator with core's composed verdicts so a
candidate that could not be read is incomplete, never a non-match. The role
list and stable-text rules live in core only; the intended visibility
promotion exposes stable_text_match and the Windows resolver composes
identity_match for the id tier with stable_text_match corroboration only when
the stored ref carries a stable text identity (KTD3). The search gates on role,
flags incomplete on an unreadable candidate or role, and the resolver returns
an incomplete-and-retryable AppUnresponsive instead of a settled STALE_REF
when everything readable found nothing. A7-3's wrong-target pin is carried
through the composition, and the blank-cannot-refute and vulnerable
mutable-value name edges are pinned both ways.
* feat: add the graded resolution fallback to the windows resolver
The identifier-free Electron shape now resolves through two graded tiers
before the broad search: a window-rooted path fast-path (path is a locator,
never identity - a miss or refuted candidate falls back to the broad search
and is never settled on), and a geometry promotion that resolves an unreadable
candidate on a unique live bounds-hash match. The promotion mirrors macOS's
predicate with the A17-7-measured addition that the stored hash must come from
a positive-area rectangle - offscreen and virtualized elements collapse to
shared zero-extent bounds and never promote. The search family is split into
resolve_search (the plan's named seam) to hold the three files under the
400-line cap. Live fixture tests prove a blank secure ref resolves through
path and geometry on an unchanged tree and the secure value reaches no error.
* feat: classify settled absence and retry only the genuinely incomplete on windows
Resolution and live reads now carry a typed complete/retryable pair over a
three-way read-path classifier (the kAXErrorIllegalArgument lesson ported): a
settled absence like not-supported or invalid-argument is never retried, a
transport or timeout failure and a vanished element are incomplete and
retryable within the deadline. UIA_E_ELEMENTNOTAVAILABLE keeps the granularity
split - settled stale for a read of the resolved target, incomplete-and-
continue for a node vanishing mid-descent, window-not-found at root. The
completed-search STALE_REF mirrors resolution_complete_absence with retryability
derived from the code default, and resolve_element_strict runs a deadline loop
that retries only the stamped-incomplete, re-verifies process liveness through
resolve_window_root on every attempt, and stamps deadline_elapsed onto the last
diagnosis on expiry rather than discarding it for a bare timeout. Classifier
arm-by-arm tests, retry call-count pins, and the redaction pin land with it.
* feat: add the shared single-element live read and the five readers on windows
One read bends all five adapter readers: a resolved NativeHandle is downcast,
corroborated against the verified process identity stamped into the handle
payload (the KTD7 change - the resolver attaches pid and generation token at
every resolution return, and a dead provider's empty-successful reads become
STALE_REF rather than a fabricated absence), then read through
properties::read_live plus the walk's own read_label/walk_vocabulary/
into_locator_evidence composition. Essential evidence (role, value, states,
bounds, actions) must all be non-Unknown or the read fails retryable
AppUnresponsive, never a partial bundle claiming completeness. get_live_value,
get_live_state, get_live_actions, get_live_element and get_element_bounds are
thin projections; enabled and offscreen read UIA's own flags, secure values are
withheld at the reader path, and FFI ad_get('bounds') stops hard-failing once
get_element_bounds answers. Live fixture tests pin each reader, the secure
withholding, and both directions of the dead-token rule; pure tests pin the
completeness gate.
* feat: resolve the locator anchor on windows for live find hydration
Selected-match hydration (find, wait --selector, materialized queries) calls
resolve_locator_anchor; it was not_supported and every hydrated find errored.
The anchor is the path-only variant of resolve_element_strict with the inverse
classification: the path is exact from the just-walked tree so there is no
broad-search fallback, and every step is descent along a stored path that
churn makes permanently wrong, so a step that lands nowhere, a role-refuted
candidate, or a vanished node settles the attempt immediately - a completed-
search STALE_REF with one attempt and the deadline intact for core's single
fresh re-observation. Only the unresponsive/transport class runs through the
bounded retry loop, which re-walks the same cheap O(depth) path. The adapter
wires the method and the settle-vs-retry split, and the live fixture tests pin
an exact anchor path resolving, a wrong child index settling stale without
resolving a neighbour, and a role-refuted landing refusing.
* docs: dogfood the windows live resolution loop against real applications
Drive the release binary's live loop against classic Notepad on a scratch
file, Explorer on a scratch directory, the WinForms and WPF fixtures and
Obsidian (Chromium/Electron), reading the JSON: snapshot, find round-trips, and
get/is live reads all answer. The WinForms fixture's stored ref re-resolves
correctly after a real WM_APP content swap (the A7-3 wrong-target defence holds
live), and Obsidian on this box presents the first-contact shell - 4 of 6
sampled shell refs go STALE_REF on a fresh client, 2 resolve - which is the
honest, judged Electron rate, while the identifier-free fallback's web-content
rate stays owned by the 2.12 shell per A17-8. Report committed with the
environment header, per-target matrix, ambiguity corroboration against U1's
census and residuals with owners.
* docs: correct phases.md and concepts for what the windows resolution work proved
Corrections in place, never annotated: the 2.5 scope's resolve_query bullet is
rewritten to the measured truth - core owns the LocatorQuery evaluator and the
adapter ships resolve_locator_anchor plus evidence completeness (KTD2); the
snapshot value slot's secure-field behaviour is split so the read side closes
here (content-free fingerprint evidence, IsPassword-gated candidate reads,
reader-path withholding) and the action side is owned by 2.6/2.7; 2.6's scope
line no longer re-budgets the enabled/offscreen ElementState fields 2.5 ships;
and the two new scope bullets record get_live_actions' lack of production
callers and that the in-web-content STALE_REF rate is measured on the 2.12
shell (A17-8). CONCEPTS.md gains Graded Resolution and Settled Absence, both
distinct from Evidence Tri-State. Shipped source comment references to plan
decision and unit ids (KTD, U, bare sub-phase numbers) are scrubbed to what is
true about the code, keeping only the ledger row citations the phase-reference
scan permits, and the FINDINGS hunk index is re-synced to the measured diff.
* style: rustfmt the resolution and live-read modules
* fix: close the code-review findings on the resolution and live-locator work
A nine-lens review with an independent plan trace and a simplification pass
found the branch red on the repo's own gates and carrying four correctness
defects. All are closed here with tests that fail when the fix is reverted.
The resolver's anchor collapsed a tri-state: a role read that failed was
indistinguishable from a role that differed, so a transient hiccup while
walking a stored path settled a permanent stale ref in the one resolver
built to survive churn. The retry loop never retried a provider's own
timeout, because that classifies Timeout while the predicate only accepted
AppUnresponsive. A window that was merely busy carried no retryable stamp,
so it read as settled and disabled every retry layer above it. And the
live read corroborated process liveness only before its property reads,
which on this build return empty success for a corpse - the reader now
re-corroborates after the reads and settles stale on a target that
vanished, instead of burning the deadline as unresponsive.
The gates themselves were the root cause of the rest. check-rust-file-size
hardcoded python3, which does not exist on a Windows developer box, so the
comment half of the gate could never run outside CI; twenty-seven comment
violations and five oversize files reached the branch behind it. The gate
now resolves its interpreter and both halves pass. The five files are split
by responsibility, the search context is bundled to fit the parameter cap,
and a UTF-8 round-trip that mangled three ledger rows is repaired.
Three scenarios the plan named but nothing exercised are now pinned: the
selector-wait poll, count-versus-materialized agreement, and the FFI bounds
path. Tracing the first disproved the plan's own framing - the selector
wait never reaches the locator anchor - and the dogfood report records that
correction alongside the dedup deliberately left unapplied.
* fix: bind a stored window ref to the process that owns its handle
A ref stored only pid + generation token and never asked whether the HWND
it carried still belonged to that process, so a recycled handle let
resolution walk another application's tree while the verified-handle stamp
re-asserted the same stored pair it had assumed. verify_stored now checks
live handle ownership through GetWindowThreadProcessId before the token,
resolve_window_root routes through that evidence instead of its own weaker
inline gate, and the window inventory propagates the verification it used
to discard. The locator anchor inherits the gate at its existing call site,
pinned by an inverted run rather than read off the call graph.
Also closes the remaining review findings on this work: a deadline expiry
mid-read now records Unknown instead of dropping unread properties, which
had been leaving the secure-field gate open; an unreadable role in the
anchor no longer collapses into a confident mismatch; and a Timeout is
retryable only when it is explicitly stamped so.
* docs: state what window identity proves and what it still cannot
The decision record claimed a recycled handle on a different process is
caught by the window gate and one reused by the same still-running process
is caught a level down by exact-evidence element resolution. The first is
true only now that ownership is verified; the second is not true at all,
because two instances of the same dialog present identical evidence, so
the sole-candidate arm resolves one of them with no geometric
corroboration. Both are corrected in place, and the residual is written
into the sub-phase that owns it rather than left in a report.
The dogfood harness is corrected the same way: its identity judgement read
role off the stored entry, which cannot distinguish a correct resolve from
a silently-resolved neighbour, and it is scoped to what the fixture can
actually show given that its ListBox exposes no items to a COM client.
* refactor: walk a resolver descent once and honour the deadline inside it
The anchor and the strict search each carried their own sibling loop and
their own path walk, so neither consulted the deadline between cross-process
calls: up to the sibling cap could run before the budget was next checked,
and one wide step could eat a whole hydration budget. The main tree walk had
had the check in the right place all along.
Both loops and both path walks collapse into one descent parameterised by
the caller's failure classifier and its expiry policy, which genuinely
differ: the search marks its list unfinished and retries, while the anchor
surfaces a timeout, because a settled miss there would tell core an element
is gone when the truth is that time ran out. Each classifier arm is now
pinned through both consumers, as is the axis wording the two resolvers
report, and the anchor draws its budget from the same helper as the search
so the bound cannot drift below the walk that issued the ref.
The sibling cap keeps reporting a truncated list as whole, now deliberately
and documented: the widest node measured anywhere is 27 children against a
bound of 10,000, the walk that issues refs enumerates under the same cap and
refuses an incomplete tree, so no ref can name an element past it, and the
unfinished flag is subtree-wide rather than per-node - marking cap-hits
unfinished would turn every correct settled miss elsewhere in the window
into a retry loop.
* test: pin the resolution gates that passed while broken
Nine gates across the resolver, the shared live read and the HRESULT
classifier had no test that failed when they were weakened. The classifier's
drift test was worse than absent: it compared the projection to the record it
is defined as, so it could not fail - proven by running a byte-identical
replica against three deliberately broken builds, including one with a
denial-class code flipped to retryable, where it passed every time.
Now pinned, each verified by breaking the exact line it guards and watching
the test fail: the live read's completeness gate, whose deletion previously
left the whole suite green; the AMBIGUOUS_TARGET branch, including that its
error stays shape-only when duplicate evidence is what produced it; the path
fast-path's fall-through on a refuted landing, which must reach the broad
search rather than settle; and every arm of the classifier, whose disposition
and retry stamp are asserted separately because Terminal and SettledAbsence
share a stamp and a flip between them is invisible to it.
The live read also stops issuing the label's cross-process calls once the
deadline is spent, which the bounded read above it already respected. The
label read is injected so a test can pin that no call is made at all; a label
that could not be read reports as failed rather than absent, since the
absence claim is one nothing measured.
Two classifier codes stay terminal with their reasoning recorded rather than
their names trusted: an enumeration's end of list cannot reach the success
catch-all, because a UI Automation error only carries an HRESULT when it is
negative; and the no-clickable-point code describes obscured or unscrolled
view state, which is what scroll-into-view exists to remedy, so settling it
would drop candidates mid-descent as a complete empty child set.
* fix: fail a probe run that did not measure anything
Four probe harnesses reported success while measuring nothing. A pass whose
binary exited non-zero wrote a placeholder capture, the run reported ok and
exited 0, and the placeholder satisfied the workflow's artifact gate - so the
guard failed by going green, which is indistinguishable from working. A pass
that never ran at all was worse: no capture, no complaint, same green.
Only a noisy failure was caught, and by accident: a binary writing to stderr
trips the terminating-error preference. A silent non-zero exit - a missing
DLL, an access violation, an explicit exit code - went straight through.
Placeholders now say they are not measurements, each harness declares the
captures a CI run must contain, and a CI-labelled run fails naming what is
missing or degraded. A dev box still records an honest placeholder and stays
green, and a genuinely negative measurement still passes everywhere, which is
the distinction the gate has to keep: recording that nothing was found is
data, recording that nothing was looked at is not. The gate lives in the
shared module rather than four times over.
The captures now survive a red run, since a failing probe's placeholders name
what did not measure and are the whole evidence trail. The artifact step no
longer errors on finding nothing, because a probe that dies before writing
would otherwise stack that on top of the real failure and read as its cause -
what asserts a run measured something is the per-probe gate, which names the
capture instead of firing only when every one in the job is absent.
* docs: give two cross-platform duplications a receiving sub-phase
Both had been carried from plan to plan since 2.3 with no owner in the
product's own scope document, which is where the next sub-phase's planner
reads and where this repo requires a deferral to land.
The resolver error payload is mirrored per adapter rather than shared: the
same details object core reads back to derive retryability, restated on both
sides. A key renamed or a value changed on one turns a retryable incomplete
into an unretried failure on one OS only, and both crates' tests stay green
because each asserts against its own constructor.
The evidence-completeness rule has the opposite alarm and is worse for it.
macOS restates core's rule clause for clause, and core's copy has no
production caller anywhere - only its own unit test - so the rule core
presents as canonical is inert. Tighten it and core stays green while the
macOS walk keeps grading by the old rule.
Both land in the hardening gate, for reasons that differ: the first needs a
core touch this sub-phase had already spent and changes the GA adapter; the
second needs no core change at all, but is a behavioural edit to the macOS
walk and so needs the macOS review, e2e and perf baseline that only that gate
runs. The second also carries a prohibition, since Windows consumes the same
requirements type and would find local reimplementation the demonstrated
pattern.
* fix: fail a probe that declares nothing to measure
The measurement gate is only as strong as its declaration. A probe that never
declared its mandatory captures - a leg added later, a refactor that dropped
the call - had an empty expectation set, so the gap check found nothing and
the run passed having asserted nothing. That is the defect the gate exists to
catch, one level up.
The check sits inside the gap computation rather than beside it, so it needs
nothing from the caller: every probe already asks that one function whether
the run measured what it promised, and a probe still carrying the old
hand-rolled tail fails on an empty declaration too. The four hand-copied
tails collapse into one shared assertion, so the reason reported can no
longer disagree with the reason that fired, and an unknown label fails the
parameter bind instead of silently skipping the gate.
Emptiness now reads as not-measured wherever it appears: an empty list, an
empty object and a blank string assert exactly as much as the absent result
that already counted as unmeasured. Values that are merely negative stay
measured - a bare list, a plain string, a zero and a false are answers, and
false is the measured negative this gate exists to protect.
* docs: let the integration gate decide rather than also implement
The hardening gate had accumulated a measurement it had to stage itself, a
core schema change, a macOS refactor bound to no platform phase, and the
release merge of the whole branch. Wanting full-branch review and having to
ship inside the release-merge commit are different requirements, and treating
them as one is what collected the rest.
The window-handle wrap rate moves to the sub-phase that builds the fixture
and registers the first interactive runner, which is the first rig that can
stage the churn to observe it. The schema addition that consumes that rate
becomes its own sub-phase, inserted after the one it depends on rather than
renumbering successors - reusing a number would leave existing references
pointing silently at different work, the way phases 1.5 and 1.6 were inserted
for the same reason. The field is required to be optional by construction,
since stored refs are serialised and a mandatory addition would turn a
corroboration fix into a fresh stale-ref failure.
The completeness-rule collapse is not a task for any platform phase but a
standing rule: a rule core owns and no production path calls is inert, and
tightening it changes nothing while the adapter that reimplemented it keeps
grading by the old one. It has a closed precedent in the accessible-name
work and an open instance now, so it moves to the cross-phase requirements
where the next platform's planner reads it.
What is left is decisions, one promotion that genuinely needs both adapters
reviewed together, verification and the merge. The two contract decisions
state the same boundary explicitly: the gate records what was decided, and a
decision to normalise ships as its own PR beforehand, because it changes
behaviour on an adapter that is already released.
* fix: carry a walk's unread regions into the resolution verdict
The path fast-path recorded whether its walk finished, and nothing read it:
the broad search declared a fresh flag beside itself, so only its own gaps
reached the verdict. A transport fault or a vanished node met while walking
the stored path therefore left no trace, and the attempt settled STALE_REF -
the element is gone, take a new snapshot - off a walk that never finished.
That path is the only tier that descends past the search's depth cap, so for
a deeply stored element it is the sole tier that can reach it at all, and its
gap is the only evidence anything went unread. One flag now spans the whole
attempt, declared before the fast path so a second one cannot be introduced
beside it.
A missing child index still settles on the first attempt, because landing
short of a stored index is a real answer rather than a failed read. What now
retries is a faulted or starved enumeration, which is bounded by the same
deadline and ends as an unresponsive app rather than a false certainty.
* fix: tell an agent what the error actually was
Three agent-facing errors said something other than what happened.
A stale ref reported "not found in current RefMap" for a ref that resolved
fine and whose live evidence simply did not match - the core constructor
takes a ref id and formats that clause around it, and both Windows sites
passed it a whole sentence, so the shipped text was also two sentences run
together. They now build the error directly, mirroring the macOS shape while
keeping the not-delivered disposition macOS omits, and both go through one
constructor rather than restating it.
A missing window root carried no disposition at all. An unstamped error reads
as retry-permitting, so a caller polling a window that was already destroyed
would spend its whole budget waiting for it. It is now stamped from the
classifier's own table at the granularity that applies: a window root that is
gone never answers the same handle again, which is settled, not merely
unavailable the way an element mid-descent is.
An ambiguous target carried no stamp either. It is complete, because the only
route to that verdict is a search that either finished or stopped where no
further candidate could disambiguate; and it is settled, because replaying
the same walk against the same stored evidence cannot answer differently.
A property read failure was the one case where the reported defect was not
real: wrapping it left the typed retryability untouched, since that is only
re-derived when the new details carry the key. The wrap now merges rather than
replaces so the classifier's pair also survives into what a caller and a trace
segment read, and the two failure families keep the separate answers they had.
* fix: catch the plan references the reference gate could not see
The gate matched decision and unit ids and phase numbers, so shipped source
citing the delivery plan in prose passed it - a doc comment saying the plan
pre-committed to a split, or naming a Verification Contract row, tells a
reader nothing they can act on and stops being true when the roadmap moves.
It now also catches the plan quoted as an authority: the possessive, the plan
paired with a prescriptive verb, a preposition leading into it, the plan
document by name, the spaced prose form of the delivery report, and plan
section vocabulary. Ordinary English is left alone - a path is a location a
reader can open rather than an authority, and a coordinate origin is not a
requirements document.
Four comments it found are rewritten to say what is true of the code: why a
prefetch split is conditional, why an ambiguous error carries shape only, why
a role and state divergence is deliberate and what it costs a caller, and why
a vocabulary walk repeats three times.
Both gates gain a self-test that runs on every invocation, sharing the rule
under test with the fixture so the fixture cannot drift from what ships.
Building them surfaced four holes: a file was measured with a newline count,
so a file whose last line was unterminated read one short and a file one over
the cap slipped through; a capitalised sub-phase escaped both the sub-phase
rule and the phase-number rule, which needs a digit; the generated-file
exemption hung on which side of a pipeline finished first; and the interpreter
probe accepted a version whose syntax error would have read as a violation.
* fix: stop a probe panicking away its own evidence
A probe that panicked wrote no capture and left its hosted fixture process
running, so the measurement gate reported a missing capture rather than the
error that caused it, and the corpus's no-surviving-process rule broke with
it. Six panicking calls are replaced by propagation or a recorded failure; a
walk that could not run now reports that it did not measure instead of taking
the process down. Driven both ways against a real provider fault.
A survival capture recorded element names verbatim. The gate every capture
round-trips checks operator identity - user, machine, profile paths, SIDs -
and has no rule for a content name, and the helper that reduces one is a
call-site reducer a Rust probe cannot reach, so nothing downstream noticed.
The names are recorded as a digest and a length, which still pairs an element
across a relaunch, and the contract is written down where the next Rust probe
will look: reduce a content name where it is recorded.
The A17 timing rows claimed more than one environment supports. The CI leg is
wired but no capture from it was ever committed, so every number rests on the
dev box alone - the third time this corpus has taken a single environment for
a settled fact. The rows now say so, separate the ordering they do establish
from the milliseconds they do not, and name what closes them.
* docs: give the mutation classifier and a mangled error an owner
The mutation-path delivery classifier was deferred to a pair of sub-phases
without either one taking it, so no scope said what to build. It lands in the
sub-phase that first invokes a UIA pattern and already owns the typed delivery
report the classifier feeds, with the outcomes and their codes named and the
rule that matters stated: the shipped read classifier calls transport failures
retryable, which for a write is how one click becomes two.
The stale-ref constructor takes a ref id and formats "not found in current
RefMap" around it. Fourteen call sites across core and the macOS adapter pass
it a sentence instead, so the message reads as two sentences run together and
blames a lookup that never happened. Windows is corrected here; the rest are
recorded against the gate that can change core and both adapters at once,
where a promotion of the same error family already opens two of the files.
The window-handle work in the observation plan still pointed at the merge
gate after the measurement and the schema change were split apart.
* fix: self-test the file-size gate without deleting anything
The gate's self-test built its fixtures on disk and removed them afterwards,
which tripped the harness contract that forbids a shell script under scripts
from deleting artifacts at all. The ban is right and the fixtures did not need
a filesystem: the cap rule reads its argument in one forward pass and never
seeks, so each fixture is now a stream. Nothing is created, so nothing has to
be cleaned up.
The interpreter probe's fixture no longer empties the search path, which was
never quite the thing under test and is awkward where a resolvable name can
carry an extension. The candidate list is injected instead, so the real
resolution loop and the real failure branch both run, and a second rejection
now has a fixture: a command that resolves but answers the version probe with
a failure, which is what an old interpreter does and what the emptied path
could never reach.
The rule stays the single owner of the cap, called by both the tree scan and
the fixture, and the fixture stays wired into every invocation. Each of the
six ways to break it was driven and observed: the cap that never fires, the
cap that rejects everything, the count that measures newlines instead of
records, the exemption dropped, and both interpreter rejections.
* docs: refresh the learnings against what the code now does
An audit of all twenty-one learnings found the advice sound everywhere - no
recommendation had to be rewritten - but nine had drifted in what they point
at, and two were wrong in ways that would mislead a reader following them.
The envelope contract's trigger list covered a success becoming an error and
not the inverse, which is what the last bump actually was: a snapshot that
exhausts its budget now succeeds and reports incompleteness in a field rather
than failing. A reader applying the doc literally would have concluded no bump
was needed. It gains that trigger and the list of surfaces a bump has to reach,
two of which were stale and are corrected here.
The gate doc justified its portability advice by saying the gates run on
several systems in CI; they run on one, and the machine most likely to author
them never executes them, which is why a pattern that worked only under one
toolchain survived review. It gains the dimension all of this week's evidence
kept pointing at: test a check's ability to run at all, because one that finds
nothing and reports success is indistinguishable from a clean tree.
The rest is currency. The private-file section is written in the present tense
about hardening that has since shipped and satisfied it, including the
technique that made one constraint tractable - prove an ambiguous sentinel by
first making a control call that must succeed. Dogfooding against real
applications is described as the standing practice it became rather than a
recurrence. The allocator doc names the one sanctioned second recursion and
draws the boundary around it. The reliability contract's verification layer is
stated per-platform now that a second platform has one.
The pointer-actions documentation gap is closed, so its doc goes; the one
observation not mirrored anywhere else - that the actionability battery probes
several points while the pointer path checks only the one it resolved - moves
to the doc that owns pointer policy.
CONCEPTS.md gains the read-outcome taxonomy, whose only defined member had
been leaning on three siblings that did not exist, and the evidence vocabulary
that separates measuring nothing from finding nothing. Entries carrying file
paths, type names, or a roadmap owner are rewritten to stand on their own.
* docs: show the snapshot envelope the shape it actually has
Four places documented the snapshot response with a data object that predates
the current envelope: they carry the right version number beside a shape from
before it. Since that version, every snapshot reports whether it finished, and
a tree too large to walk succeeds and says so in a field rather than failing
with a timeout - so an integrator following any of these examples would branch
on an error that no longer arrives.
The C header is regenerated from its source rather than edited, and the two
examples that were also missing the snapshot id gained it: a shape known to be
wrong in one field is not worth publishing to be right in another.
Left alone deliberately: the error envelopes, which the version did not
change, and the entries that assert nothing about the data object. The
traversal timeout row stays true, because a drill-down from a stored ref still
returns one - it observes a subtree whole or not at all, so it never reports a
partial result the way a full snapshot now does.
* test: measure the adapter's own withholding, and the gate that guards it
The secure-control probe read the provider directly, and the provider blanks
those slots itself - so the arm passed for the platform's reason and would
have kept passing with the product's withholding deleted. It now takes two
readings of the same control: the provider's, whose recorded numbers are
unchanged, and the adapter's own composition. The pass is gated on the
provider having published something first, because an adapter that published
nothing is otherwise indistinguishable from a provider that offered nothing.
One target publishes no readable slot at all, and that pass now says so
rather than counting itself as evidence.
The reader test gains the same treatment. A marker pin alone would not have
caught this, because the marker never reaches the slots the provider blanks;
what catches it is comparing what the platform offered against what the
reader passed on.
The measurement gate - the check that fails a run which measured nothing -
had no self-test of its own, so a regression returning "no gap" would have
let every run report success after asserting nothing. It now carries one that
runs whenever the shared module is loaded, drives the shipped functions
rather than a copy of their rules, and covers both directions: a placeholder,
a pass that never ran, an empty declaration, and the negative result that
must still pass. Building it exposed two holes in the fixture itself, one
silent - the decision was extracted so a test can reach it without ending the
run, and the alarm is now read a second time independently, because an alarm
routed through the circuit it monitors is not an alarm.
The geometry promotion arm had no test that failed when it was deleted. The
per-node decision is extracted so both halves are pinned: an unsettleable
identity promotes when a positive-area rectangle agrees, and a zero-extent
one never promotes however exactly its hash matches.
* test: make the checks fail when what they guard is broken
Five review rounds each surfaced one more assertion that would survive its
own subject being deleted, so this sweeps the class instead of the instance.
Every test in the Windows crate was read and every claim below was proven by
breaking the exact line it guards and watching what happened.
Twenty assertions survived a mutation and now catch it: a sentinel whose
disposition could be flipped, six state producers that could each be deleted,
a property that could take another's name, two retry arms, a modal check whose
only test ran the non-Windows stub, an evidence projection whose two sources
were seeded identically and so were indistinguishable, two private-file gates,
a promotion failure tested as a formatter but never as a wired path, an action
vocabulary asserted against a superset containing names the producer cannot
emit, and three redaction rules.
Two lines guarding real defects had no test at all. Deleting the resolver's
verified-process stamp disarmed the corpse check for every resolved handle,
and turning the bounds-hash tie-break into first-match-wins silently resolved
the wrong element among hash-matched candidates - the shape measured on a real
file list. Both are pinned against fixtures where the right answer is not the
first one.
One production defect: the tree dump redacted only the first user path in a
value, and a provider chain carries one per provider, so a second path was
written out verbatim. Every occurrence is consumed now.
Eleven further lines are proven unguarded and named in the report rather than
quietly left; two pins cannot be inverted because the property they assert
comes from the live system, and they say so.
* fix: stop the gates and probes reporting what they never measured
The pre-commit hook had stopped running any gate. It selected staged files
with a pipeline whose reader quits at its first match, so the writer died of
a closed pipe and, with pipefail set, finding a match became a non-zero
status - a match read as no match. Once the staged path list outgrew the pipe
buffer the hook announced there was nothing to check and exited clean. It
reads the list once now, and both directions are driven against a list far
larger than any buffer, because this failure announced itself as success.
The npm release gate searched the whole workflow for a publishing permission
that another job also holds, so deleting the grant from the job it guards left
it passing. The bash-3 gate rejected two spellings of an associative array and
silently accepted three other constructs from later shells. The comment
scanner's own tests were named so the lane that discovers tests could never
match them, so they had never run. Each now carries a fixture that shares the
shipped rule and runs on every invocation, in both directions.
Among the probes: a comparison of two joined lists always evaluated to a
truthy string, so every path-survival and bounds-equality field reported
agreement for values that differed. An anti-vacuity guard counted a null as
one element and so never fired on the one input it exists to reject. A
registry lookup used a drive that does not exist, making a published
registration answer unconditionally false. A list-swap classification was
published from runs where the mutation never landed, and a leak check planted
its secret without reading it back, so deleting the plant changed nothing it
reported. A private-file probe published two dozen rows about renaming over an
open handle without ever opening one.
The redaction gate reached three quarters of the committed captures; it now
reaches all of them. A document title that lands on a container escapes the
content rule, because a container takes its name from its contents and the
window title only tracks the active pane - one such title was published and is
withdrawn. Chrome labels stay verbatim: the rule reduces only a name echoing
something already reduced beneath it, inside a target declared to carry user
content, which the unscoped form was measured to get wrong.
Three ledger rows are corrected to what re-measurement shows.
* fix: let the pre-commit hook run on the platform this phase targets
The hook drove clippy and the library tests across the whole workspace, which
resolves through default members and so pulls in the macOS crate. That crate
cannot build anywhere else, so the hook failed outright on every Windows and
Linux machine and the documented way to commit was to skip it. Skipping it is
how a formatting break and a batch of comment-rule violations reached CI this
week: the gates that would have caught both run in the hook.
Both invocations now take the package set the host can actually build, taken
from the CI lanes rather than invented - Windows and Linux mirror their own
lanes, and macOS keeps everything it already covered so the released platform
loses nothing. A guard refuses the commit if the macOS crate ever enters a
non-macOS set, since that is the failure being fixed. Format stays workspace
wide because it builds nothing.
Everything else is untouched, including the staged-file matching whose own
fixture still fails in both directions when sabotaged.
* docs: correct what actually gates a chromium content tree
Two scope entries recorded that a web target's tree only reaches a client
that settles long enough, and that the dev box could therefore only ever see
a first-contact shell - so the hardening sub-phase was told a longer-lived
client on its own runner was the lever. Measurement disagrees.
On this dev box a restored Chromium target answers the product's own path
immediately with a complete tree and allocated refs, while the same process
minimised answers with a timeout and shows a raw walk eighteen nodes. Holding
one client against the minimised window for ninety seconds adds nothing, and
neither does the accessibility handshake the window answers: the tree is
wholly there on the first read once the window is restored, and wholly absent
while it is not. Window state decides it, not elapsed time.
The rate inside web content is still unmeasured and still owned by the
hardening sub-phase. What changes is that it starts from a true premise, and
its first obligation is to control the target's window state rather than to
attribute a thin tree to the machine.
* refactor: split the two facts one resolution flag was carrying
A single flag told the resolver both that an attempt had left some region of
the window unread and that a particular landing was uncertain. Those are not
the same claim, and for a landing at a stored index they are provably
unrelated: the child list is only ever appended to, so a truncation drops a
suffix, and a landing that resolved an index at every level had every read it
needed. Whatever went unread lies past it.
Carrying both facts in one flag meant each reader saw the one they cared
about. One reading found the flag discarded and concluded a missing element
could be declared gone off a walk that never finished. Another found the same
flag reaching a successful landing and concluded the landing was unsound.
Both were reading a value that could not answer either question on its own.
The path walk now returns its own landing, carrying whether the walk left a
region unread as a property of that walk rather than as an argument threaded
into the tier below. Accepting a landing is a separate function that has no
completeness parameter at all, so the tier cannot be made to second-guess a
positional answer with a fact about elsewhere. The verdict takes the landing
itself, so the walk's gap cannot be dropped on the way without writing a
claim that no walk ran, next to the walk that ran.
The property this protects was never actually pinned: reintroducing the
discarded-gap defect left the suite passing. Both directions are pinned now,
along with the prefix lemma the positional answer rests on, and the tier is
load-bearing rather than redundant - a control with no settleable text
identity resolves through it and cannot be resolved without it.
|
||
|
Lahfir
|
bbad7721fb
|
feat: windows observation read path — snapshot, inventories, chromium settle, drill-down (#119)
Lands the full Windows read path: observe_tree wired to the shipped walker with honest completeness (partial trees, liveness-verified complete claims, budgeted boundary counts), HWND-first window identity corroborated by a process-generation token, list-windows/list-apps/list-displays inventories, focused-window composition, the four P2-O8 descriptor fields as absent-by-default cross-platform schema, Chromium detection with a deadline-bounded activation settle (--timeout-ms, --force-electron-a11y), the provenance-gated web-wrapper depth-skip, and fail-closed drill-down resolution. Verified by a nine-reviewer code review (twelve validated findings fixed with revert-sensitive tests, two rejected with evidence), a two-wave post-fix dogfood against real applications including cold-start Chromium, and a two-environment cost corroboration (A16 ledger). |
||
|
Lahfir
|
a2712f18a5 |
chore: merge main into the windows adapter branch
Brings 0.7.0 and the observation and action fixes (#117) onto the Windows integration branch so the promotion PR is no longer conflicting. One conflict, in CONCEPTS.md, and it was additive on both sides: main added Partial Observation, this branch added the Vocabulary section. Both are kept — Partial Observation belongs to Desktop Observation, beside Drill-down, which is where it now sits, and Vocabulary follows it. |
||
|
Lahfir
|
32175e44c5
|
fix: return observed trees and stop demanding renderer activation from shallow walks (#117)
* fix: return observed trees, stop demanding activation from shallow walks Six defects found by measuring the observation and action paths against real applications. Each was code computing something correct and then discarding it. Snapshot discarded a fully-observed tree when its budget expired. Finder observed 247 nodes, threw away all of them, and returned TIMEOUT with zero refs. It now returns what it observed with `complete: false`, and every node whose descendants were cut carries `subtree_truncated`, which propagates to its ancestors so a reader can walk from the root to the boundary. Only a full snapshot may be partial: a drill-down replaces refs inside an existing map, so it still requires a complete observation rather than destroying descendants it may not be able to re-allocate. `kAXErrorIllegalArgument` was classified as retryable. It is the window bridge rejecting an element outright, which no amount of retrying will change, so strict resolution burned its full budget on a call that could not succeed and then blamed the application. Finder refs went from 0 of 9 resolving at ~814ms to 6 of 7 at ~120ms. A depth-clamped observation was treated as evidence that a renderer had no accessibility surface. A shallow walk stops above the web content by design, so every skeleton snapshot of a Chromium application demanded an activation it did not need and re-walked the tree until the deadline expired. Slack `--skeleton` went from failing at 3.4s to 0.2s, and depth now scales monotonically instead of shallower being slower than deeper. The retry that follows a genuine activation also backs off, because each attempt costs a full tree walk: a 3s budget spent about 120 of them and now spends 11. A boundary node is read for its child count alone. On a renderer that materialises children lazily that count can cost more than the traversal it describes, so it is now best-effort; a boundary that cannot afford one is still reported as truncated, just without a number. Refmap retention kept 512 snapshots and swept every one of them for orphaned temporary files on every save. Retention is now 128 with eviction to 96 so the sort-and-stat pass is amortised, the per-save sweep covers only directories that save could have written, and the exhaustive sweep runs with eviction. `session end` drops the ref scaffolding it accumulated, but only under `ArtifactsMode::Full` where the trace keeps its own copy; the default mode never copies them, and discarding them there would sever snapshot resolution for anyone reading the trace after. `is --property` read element bounds for properties that never use them. BREAKING CHANGE: ENVELOPE_VERSION is now 2.2. `data.complete` is present on every successful snapshot, and a snapshot that exhausts its budget returns `ok: true` with `complete: false` where it previously returned a TIMEOUT error. Callers that branched on TIMEOUT to detect an oversized tree must read `complete` instead. * docs: correct solution docs that contradicted the code and each other A refresh pass over docs/solutions/ against the current tree. Three of the four corrections were internal contradictions that reading the doc alone could not reveal. The pointer-action doc cited `resolve_point_with_deadline`, which has no matches in the repository; hover and drag resolve in two phases, before and under the interaction lease. It also now distinguishes the two hit-tests that exist, since the shared battery's multi-candidate check has grown to cover the click family while the pointer pipeline keeps its own single-point check. The drag-abort doc claimed the release guard "arms only after mouse-down is posted". It arms before, which is what the doc's own prevention rule requires and what the code does. The progressive-snapshot contract described one truncation path. There are two, and the second was silently dropping descendants with no marker at all, violating that doc's own rule. CONCEPTS.md gains Interaction Lease, Partial Observation and Delivery Semantics, and corrects Interaction Policy, which claimed ref commands expose exactly two modes when a third is the base policy of an explicit key press and is directly selectable by language bindings. * fix: only discard refmaps the trace actually copied Being in ArtifactsMode::Full was treated as proof that every refmap had been duplicated into the trace, so ending a session deleted the whole snapshot store. It is not proof. The artifact byte budget rejects a copy once a session's refmaps exceed it, and a serialisation failure skips one too; both report success to the caller. A long full-artifacts session therefore holds snapshots whose only refmap lives in the store, and ending it destroyed them permanently, severing snapshot resolution for anyone reading that trace afterwards. Each snapshot directory is now removed only against its own duplicate in the trace, and the latest-snapshot pointer survives unless the snapshot it names is gone. The existing full-artifacts test encoded the defect: it seeded a snapshot with no trace copy and asserted the store was emptied. It now seeds the copy it claims exists, and a new test covers the case that was losing data — a refmap the trace never copied must survive. |
||
|
Lahfir
|
8f24f04f5a
|
feat: windows vocabulary — roles, states, native_id and name evidence (#115)
Gives the Windows UIA tree its vocabulary: a ControlType→Role map with no catch-all, available actions, a gated state vocabulary, native_id from AutomationId, and name evidence resolved through core's single shared precedence. Core is touched exactly twice; macOS output is byte-identical. Pattern state is read as plain batched properties, each gated on its own Is*PatternAvailable flag — a provider returns a plausible default for a pattern it never implemented, so an ungated read decorates every inert node. invalid and pressed are deliberately unproduced on Windows: the first has no source that is a positive claim, the second no reachable role precondition. Evidence is tri-state throughout. Absent is an answer and Unknown is the lack of one, and the two are never collapsed — a failed read withholds a role, state or affordance rather than granting it. Includes the probe corpus and findings ledger behind these decisions, a census tool that reports shapes and counts without ever serializing a real application's text, a dogfood run against four real UI stacks, and a gate that fails the build when shipped source references the delivery plan. |
||
|
Lahfir
|
41fc178e5e
|
feat: uia element wrapper & tree walk (sub-phase 2.2) (#114)
Fills crates/windows/src/tree/ with the four things every later observation sub-phase consumes: an element wrapper with delegated refcounting, a timeout-bounded UIA client and window-root resolver, a tree walk with a cycle guard and honest completeness, and provider-class-conditional CacheRequest batching. Ships the secure-field gate, an out-of-process fixture, committed COM censuses as dev-box evidence, and corrections to docs/phases.md. Measurement changed the design repeatedly, and each result is a ledger row rather than a silent correction: - A14-3: the end-of-list discriminator is not inverted - code() 0 with result() None, on both builds. - A14-4/A14-9: a dead provider is invisible on both the sibling and property axes, so neither carries liveness. Recorded for 2.4, which owns the snapshot path. - A14-8: IsOffscreen disagrees with itself inside one minimized window, contradicting A1-2 off the managed stack. - A14-10: 2.1's owned-process COM bootstrap is thread-local behind a process-wide guard. - A14-11/A14-12: ElementFromHandle against a non-dispatching window hangs rather than timing out, and the bound that stops it is reachable only from CUIAutomation8. There is no fallback to an unbounded client. - A14-13: UI Automation's client core initialises lazily and not re-entrantly, so concurrent first touches fail with E_FAIL. The accessor serialises that first touch. Every control was observed failing before being claimed: the cycle guard, the discriminator, the secure-field gate, error redaction, capture normalisation, fixture teardown and both hang bounds. ObservationOps::observe_tree still returns PLATFORM_NOT_SUPPORTED; no ref is allocated anywhere in crates/windows. |
||
|
Lahfir
|
18daaa8215
|
feat: windows toolchain, ci & com bootstrap (sub-phase 2.1) (#112)
* docs: correct the windows sub-phase facts research disproved Sixteen replacements, no annotations. The document is the product's source of truth and now reads true. The COM bootstrap is split by consumer. CoInitializeEx is right for the CLI and wrong for the cdylib, which fails RPC_E_CHANGED_MODE against any host thread already in an STA and whose per-thread balance can never be released from a Drop on another thread; the library path uses CoIncrementMTAUsage, whose cookie is thread-agnostic. MTA is recorded as a requirement rather than a preference, because the single-threaded apartment can prevent handler removal outright. Two things the sub-phase asserted were not reachable. Its private-file seam named a boundary that does not exist: every write site is in core with no adapter handle, and core may not depend on the platform crate. The document now states that constraint and leaves the mechanism to the plan. Its CI paragraph claimed all runners enforce clippy, isolation, and the size cap; only the macOS job does, and the other two lanes each run a single lib invocation. The event invariant said handlers are drained on the dedicated MTA thread. They are not: delivery arrives on several UIA-owned threads at once, the registering worker among them and the main thread never, so handler state has to be safe for concurrent delivery. Runner registration gains the hardening a public repository requires, and gains the measurement it already owed the ledger: it closes the deferred RDP row rather than merely documenting session isolation. The DPI bootstrap records that awareness cannot be verified by read-back, since the query API has no enumerant for the value being set. The ledger's hunk index moves with them, 37 to 40. Hunks backed by research rather than by a probe are labelled as such, so an external citation cannot pass itself off as a measurement. * docs: move the windows runner registration to the sub-phase that needs it Sub-phase 2.1 was to register a self-hosted interactive Windows runner that nothing in 2.1 through 2.11 uses. On a public repository that is a persistently registered runner sitting idle behind ten sub-phases, and GitHub's own guidance is that self-hosted runners should almost never be used for public repositories at all. Registration moves to 2.12, the first sub-phase whose gate needs a real desktop, and it takes the hardening requirements with it: dispatch-only triggering, a written fork-PR approval policy, and an explicit ephemeral-versus-persistent decision. The deferred RDP row moves with it. Its closure was never documentation -- it is a measurement that only exists once a second, non-console session does, so 2.12 now owns closing it rather than 2.1 owning a promise it could not keep. Two probe scripts hardcoded the old closure sub-phase in text they emit. The scripts were corrected and their probes re-run, so the captures were regenerated rather than edited by hand. * docs: plan the windows toolchain, ci, and com bootstrap sub-phase Eight units covering the CI lane extension, Win32 bindings in the platform crate, the COM and DPI bootstrap, the adapter session, the permission probe, and a private-file layer rebuilt from scratch against sub-phase 2.0's measured evidence. Three reviewers found what the first draft got wrong. It installed both the apartment and the private-file seam from ad_init, which the FFI header labels optional and whose own doc promises it initialises nothing -- so any host that skipped it would have run every private write through the unhardened default with nothing to distinguish the two states. The install moves to the site the macOS arm already uses, where a consumer cannot skip it. It also planned a layer that validates security state it never authors. The deleted implementation at least built an owner-only protected descriptor at creation; the rebuild now does too, and rejects reparse points per component the way the unix path rejects symlinks. The ACE validation order was still wrong in the same shape as the defect it exists to prevent: checking the type before forming the SID pointer still dereferences the header at an offset nothing has bounds-checked. The order is now extent-first, with the ACE count treated as advisory rather than trusted. The seam routes at core's five private-file primitives rather than at a list of call sites, which covers the lock and liveness paths a list had missed and needs no call-site edits at all. * docs: cut acl validation from the windows private-file design Reverses the ancestor-walk decision on measurement. A plain leaf under the user profile already inherits SYSTEM, Administrators and the user with no Users entry, all inherited and none explicit, so the walk would verify a property Windows already guarantees. Unix has to author its permission because the default there is world-readable; Windows does not, because the default is already private. The only principal the walk would additionally exclude is another administrator, who holds SeTakeOwnershipPrivilege and can seize any object whatever its DACL. Building a chain-walking validator against someone who can bypass it costs more than it buys, and the cost is specific: parsing ACEs is what the previous layer did, and its AceSize handling is the defect that took the layer down. Removing the parser removes the bug class rather than fixing it, so no successor module exists and a test asserts the ACE symbols never come back. What survives is the control with real unix parity and a real attack behind it: per-component reparse-point rejection. That is an integrity control, not a confidentiality one -- a junction planted on the path redirects where the product writes, which no permission on the intended destination prevents. Cross-admin isolation is now answered rather than open: out of scope, for the same reason macOS never raised the question. * test: resolve the test binary per platform via CARGO_BIN_EXE * feat: add target-gated windows-sys bindings to the windows crate * feat: route private-file primitives through a core-defined platform seam * feat: establish com apartment and dpi awareness at unskippable call sites * feat: report windows permissions truthfully from a host-independent seam * ci: extend the windows lane to the full package surface * feat: give the windows adapter a session that owns apartment lifetime * feat: rebuild windows private-file hardening behind the core seam * test: pin the inherited-acl assumption module-free on a real profile leaf * feat: install the windows private-file seam at both unskippable consumers * fix: scope the temp lease per write and fold duplicated windows helpers * fix: route windows session and trace directory creation through the private-file seam * fix: scope the com bootstrap to adapter commands and harden review-flagged tests * test: cover the owner, locality, replace-acl, and sweep-race branches of the windows hardening * fix: pin the private-file directory chain to close the junction-swap window * docs: settle the user-output seam exclusion and pin it with a test * refactor: split the trace file plumbing and ffi adapter tests into sibling modules |
||
|
Lahfir
|
5fd9543a79
|
feat: add windows platform exploration probe corpus (phase 2.0) (#111)
* docs: add the sub-phase 2.0 probe corpus plan The plan for the Windows platform-exploration sub-phase: an eleven-area probe corpus, its findings ledger, and the same-PR phases.md corrections that ledger obliges. Lands on the 2.0 sub-phase branch so the sub-phase PR reviews as one self-contained unit. * feat: add windows probe harness with redaction gate and integrity helpers The harness is the foundation every 2.0 probe codes against: common.ps1 owns the R11 redaction gate, BOM-less UTF-8 capture writers, KTD9 normalization, scratch-process lifecycle with pid tracking, the KTD5 foreground assertion, and a Medium-integrity process launcher. runas.exe /trustlevel:0x20000 was measured on this box and produces a High integrity process (S-1-16-12288) with a deny-only Administrators group, not the Medium label the UIPI probe needs. Start-MediumIntegrityProcess therefore duplicates the token and sets S-1-16-8192 explicitly, asserting the label on read-back so a UIPI row can never be graded against a fake boundary. * feat: add winforms and wpf scratch fixtures with explicit automation ids The fixtures are the interaction targets for the pattern census, identity stability diff, raw interaction exercises, and the COM shim's event sources. Two measured platform facts shaped them. First, the .NET Core rule that a WinForms control's Name becomes its UIA AutomationId does not hold on .NET Framework 4.8 -- the HWND provider supplies raw window handles instead, which change every launch and are useless for an identity-stability diff. The fixture installs a server-side provider per control, with a switch to fall back to explicit Win32 control ids. Second, Win32 UIA client-side proxies are inert on this VM: the WinForms fixture exposes 23 controls with zero patterns, every one a Pane, matching stock Notepad's Edit reporting as a bare Pane. The WPF fixture exposes 25 controls with a full pattern surface. Interaction probes therefore target WPF and record the WinForms absence as evidence rather than working around it. * feat: capture windows environment, session, and integrity baseline Every ledger row inherits these machine facts, so they are captured reproducibly rather than asserted: build 17763.7434, session and interactivity, mandatory integrity label, UAC policy state, default file ownership, display topology, and tool inventory. Identity is recorded as a shape, not a value. The account SID keeps its well-known RID tail (S-1-5-21-<redacted>-500) because the RID is the load-bearing fact, while integrity and well-known group SIDs stay verbatim for the same reason. Three findings the later units depend on. The GAC UIAutomationClient assembly exports 57 types, all managed System.Windows.Automation, and no IUIAutomation -- while CUIAutomation8 is registered against uiautomationcore.dll, so UIA3 is reachable only through a compiled interop shim. Newly created files are owned by S-1-5-32-544 via TokenOwner rather than TokenUser, pre-answering half of the private-file ownership question. The locale is split: UI culture es-ES while the current culture is en-US, so OS strings are Spanish but .NET number and date formatting is not. * feat: probe windows private-file primitives for the 2.1 hardening design Answers the four questions sub-phase 2.1 rebuilds its private-file layer against, measured on the real OS with verbatim Win32 error codes. Three of the four contradict the framing this work was planned under. Rename: MoveFileEx cannot replace a target that has any open handle, and it reports ERROR_ACCESS_DENIED rather than the sharing violation the plan assumed -- FILE_SHARE_DELETE does not rescue it, because MoveFileEx issues ReplaceIfExists rather than POSIX-semantics rename. ReplaceFile is the operation that honors share-delete on the destination, but it is the one that refuses an open handle on the replacement source. The two APIs have opposite tolerances on opposite sides, so code treating error 32 as the sharing case misclassifies the most likely production failure. Ownership: new files take their owner from TokenOwner, never TokenUser, at both High and Medium integrity. Integrity level is not the variable -- the default follows admin-group membership, so an owner-only DACL check breaks for any admin account regardless of elevation. Locality: GetFileInformationByHandleEx(FileRemoteProtocolInfo) does distinguish local from remote, but signals local by failing with ERROR_INVALID_PARAMETER rather than succeeding with a local protocol value. An out-of-range info class returns that same error, so the code is ambiguous and a control call is required before reading it as a locality signal. ACL: neither leaf carries an explicit ACE. The restriction is authored at the nearest protected ancestor and merely propagated, so a leaf-only check sees the effective grants but cannot tell whether they are anchored or silently re-derivable, and cannot see FILE_DELETE_CHILD on the parent. * feat: probe uia3 com for events, caching, walkers, and pattern availability This is the stack the Rust adapter actually wraps, and it does not agree with the managed client. Notepad's edit control is a bare Pane with no patterns to System.Windows.Automation but a Document with Value, Text, Text2, Scroll and LegacyIAccessible to UIA3 COM, which names the responsible EDIT proxy in ProviderDescription. Twenty-six nodes versus three. Win32 client-side proxies are alive here; they are inert only for the managed client, so a managed census would have recorded false absences for patterns the adapter will see. The scratch fixture's earlier zero-pattern reading was a fixture artifact rather than a platform fact: a custom server-side provider suppresses both client-side proxies and WinForms' own providers. Both fixture modes are censused so the ledger states the mechanism instead of the symptom. CacheRequest measures 2.73x against the documented 3-5x, but the headline number hides the shape: building the cache makes the find phase slower, and the entire win lands in the read phase at roughly 299x. The claim is wrong in both directions. Handler teardown is safe with events in flight, costing 69ms against a backlog of 281. The hazard is elsewhere: hiding and showing a top-level window while handlers are registered poisons subsequent removal for up to 86 seconds, superlinearly in repetitions, and cheap callbacks do not avoid it. A watch implementation needs a teardown timeout rather than an assumption. Pattern ids are discovered from the OS at runtime rather than hardcoded, which caught that the annotation availability property is 30118. * feat: dump notepad, explorer, settings, and obsidian trees with real geometry Targets launch restored-but-not-activated, so bounds are real: 100/95.9/100/99.2 percent non-empty across the four dumps, with focus asserted identical before and after all seven walks. Settings resolves through ApplicationFrameWindow to its CoreWindow's SystemSettings pid, because the frame window belongs to ApplicationFrameHost and no predicate in either script keys on a display name. Minimizing a window does not shrink its tree -- the node count is identical at 3 -- but it degenerates geometry in two different shapes. Only the top-level window reports an empty rect; its descendants report real dimensions anchored at -32000, and every node still reports IsOffscreen as false. An occlusion gate testing emptiness or IsOffscreen alone would accept both as visible. Chromium activation is not a client-stack difference. The managed client takes Obsidian from 9 to 119 nodes across an 8 second settle with no flag, matching the COM shim's shape. What suppresses it is occlusion: an earlier revision that left other targets restored on top held the count at first contact for a 16 second instrumented hold across three runs. The mechanism is not isolated, so it is recorded as a probe-placement hazard rather than a product claim. The managed timing cross-check reproduces the COM stack's direction but not its magnitude, and on Notepad caching is a pessimization at 0.21x, because client-side providers run in the client process and the cache is pure overhead. * feat: census patterns and automation ids, and measure identity survival Identity survival is what sub-phase 2.5 designs its Windows RefEntry against, and the three properties behave nothing alike. AutomationId survives process restart on every stack measured, at 100 percent across WinForms, WPF, and real Win32. RuntimeId survives restart nowhere, at zero percent on all three. Path survives restart perfectly and is the first thing a content change breaks. Under content mutation the failure is worse than loss. Explorer keys list rows by row index, so after a folder changes, 29 AutomationId keys still resolve and 5 of them land on a different file. A ref keyed on AutomationId alone does not fail -- it succeeds against the wrong element. That silent-wrong-target count, not the survival percentage, is what makes stable text identity load-bearing. Measuring it needed a real refresh wait: a 4 second settle reported a wholly unchanged tree and would have recorded Explorer identity as perfectly stable, the exact inverse of what the window does once it refreshes at 20 seconds. Two stack facts. WPF automation-peer binding is a one-shot race -- a client that reads before the peer exists binds the generic HWND provider and never re-resolves, which a 30 second poll never recovers. And the managed and COM clients report different AutomationId values for the same window, so the divergence between stacks is in values as well as in visibility. The honest pattern divergence is two, not eleven: LegacyIAccessible and Text2 are the only patterns the managed stack structurally cannot name. The larger figure was an artifact of the WPF peer race and was retracted after the fix. * feat: probe uipi, dpi awareness, and chromium activation UIPI is measured across a boundary that actually exists. This box runs the built-in Administrator at High with Admin Approval Mode off, so RunAs yields High-vs-High and runas /trustlevel leaves the mandatory label untouched; the probe manufactures a real Medium token instead and refuses to record a verdict unless the worker reads back S-1-16-8192. Both arms run identical code against the same High Notepad, so the High arm is a control proving the injection works. UIA reads succeed from Medium; SendInput does not land, and the two arms return the identical success code -- only re-reading the target separates them. PostMessage is denied with error 5 while SendMessage WM_GETTEXT succeeds, so reads cross the boundary and writes do not. The 125 percent DPI measurement is not achievable here and the reason is measured rather than assumed: the VMware display carries no EDID and offers a single scale step. DisplayConfigSetDeviceInfo returns success and persists the registry value while effective DPI stays at 96, which is itself worth recording -- that API's success return is not evidence the scale changed. The non-zero delta defers to 2.4, which owns per-monitor scale. What does land for 2.1 is that PER_MONITOR_AWARE_V2 succeeds on 17763 with both arms verified at the awareness levels claimed. Chromium activation is graded, not guessed. The bundled version was read off this installation by scanning the shipped binary's user-agent string: Chromium 142, Electron 39.8.3. Both stacks reach the same settled tree with no flag, so the exposure claim holds and the plan's candidate contradiction is superseded. The flag is still not redundant: without it first contact is deterministically the pre-activation shell, and with it first contact is a race. Neither arm removes the need to settle before trusting a first snapshot. * feat: exercise interactions, input synthesis, and hit-testing raw Every interaction records pre-state, action, then an independent re-read -- never the call's return value. Toggle proves why: TogglePattern flips WPF ToggleState without raising Click, so the fixture's status sink stays silent while the element itself changed. Trusting the sink alone would have recorded a failure that did not happen. Posted keystrokes are not uniformly dead. WM_KEYDOWN posted to a Win32 edit control does register, because TranslateMessage runs in the target thread's own pump and synthesises the character regardless of how the message arrived; what the path cannot carry is modifier state. Against Chromium it does not register at all, with every call still returning success. Establishing that needed an idle control pass: the Chromium tree moved on its own between two reads, and without a quiet baseline that drift would have been filed as a keystroke landing. The astral-plane payload survives typing. SendInput forces a surrogate pair into two separate unicode events and the target reassembles it intact, read back through WM_GETTEXT so the check is independent of both the injection path and UIA. Notepad's edit control is unreachable from the managed client, and not merely unenumerated: TryGetCurrentPattern returns false for Text, Value and Scroll on the handle-resolved element, while the COM census sees the same window as a Document carrying all three. Hit-testing is the only sound visibility primitive of the three tested. The zero-size control is addressable by handle, enumerable by no walk, and returned by no point. A minimized window reports an empty rect at the top level while its descendants report real dimensions at the -32000 anchor, with IsOffscreen false throughout -- so neither emptiness nor IsOffscreen can gate occlusion, but ElementFromPoint correctly returns the occluder. * feat: add findings ledger, session evidence, and phases.md truth-sync Seventy-nine rows across all eleven evidence areas, each carrying the stack that produced it and whether it generalizes, so later sub-phases can tell an API-contract fact from an observation about this box and these app versions. Forty-three confirm the roadmap, five contradict it, twenty-six are new edges, and five defer to a named Phase 2 sub-phase. No row is unknown. Six in-place corrections land with it. The CacheRequest multiplier is replaced by its phase decomposition, because a single number is wrong in both directions. The private-file requirements now name ReplaceFile rather than MoveFileEx for a replace over an open destination, record that share-delete is necessary but never sufficient, and permit locality inference only behind a control call, since an out-of-range info class is indistinguishable from a local volume by error code. The Chromium guidance now requires a settle before concluding a tree is thin; its exposure claim was left standing because the probe confirmed it. The event-teardown mitigation now names the hazard that actually dominates removal cost. The completeness check is a script rather than a checklist, and it is negative-tested: blanking a scope, setting a verdict to unknown, or dropping a closure point each fail it loudly. It asserts the hunk index is bijective against the measured diff rather than against a number written in prose. * fix: wake a suspended settings instance instead of failing the tree dump The probe skipped its own launch when SystemSettings was already running, then waited the full window for a frame that a backgrounded UWP instance never presents, so a second harness run failed where a standalone run passed. That is the plan's recorded process-lifetime suspension risk arriving in practice. After eight seconds without a frame the probe now issues ms-settings: to wake a suspended instance, and the failure text names suspension as the cause rather than reporting only that no frame was found. * fix: close four teardown and gating defects found by review None of these changed a measurement; all four were failure paths that could report success. The COM shim killed Obsidian by process name with a raw Stop-Process inside an empty catch, so those pids never reached the ledger. A survivor there was invisible to both the probe's own teardown and the harness survivor check -- the one place in the corpus a leaked process could go entirely undetected. Obsidian pids are now registered and terminated through the confirming path. The tree dump registered its Settings pid only after frame resolution succeeded, so a resolution timeout threw first and leaked a process the probe had launched. Registration now happens as soon as the pid exists, and only when the probe launched it, since killing a pre-existing instance would be the operator's window. The modifier-release sweep injected without a foreground assertion, the only unbracketed SendInput site in the corpus. It now asserts and logs rather than aborting, because it runs during teardown. The input probe swallowed a failed teardown capture write, so the file that proves clipboard, cursor and process cleanliness could vanish while the probe still reported ok. It now fails loudly, and a surviving process downgrades the status too. Separately, a focus change is now classified rather than assumed hostile. A target the probe launched taking focus during its own startup is the target activating itself; Electron does this asynchronously when its renderer becomes ready, so under load it lands mid-dump and failed the run intermittently while passing standalone. Only a focus move to a window the probe never launched is interference. * chore: retrigger ci after an unrelated interaction-lease flake The macOS lib-test job failed on interaction_lease::tests::home_changes_do_not_change_the_physical_interaction_lock, which timed out acquiring the physical lock inside a 100 ms deadline at a contention count of 14. This branch changes no Rust, and the other 975 tests in that job passed, so the failure is runner contention rather than anything in the probe corpus. * chore: stop committing normalized capture twins Every capture was committed twice: once raw and once with pids, handles, timestamps and bounds canonicalized. The twin is produced by common.ps1 on every run and is byte-reproducible from the capture beside it -- verified across all thirty-seven of them -- so tracking both doubled the evidence diff for no information a re-runner cannot regenerate locally. The twins are still written on disk and -Compare still diffs them, so re-runnability is unchanged. |
||
|
Lahfir
|
4fa7661608 |
feat: open the Windows adapter phase with corrected platform facts
Establishes feat/windows-adapter as the base branch for all Windows work. Sub-phases 2.0 through 2.15 are cut from here and merge back here; main stays the macOS-GA line and gains Windows once, at the end, when the adapter is production-solid as a whole. Corrects the Phase 2 Windows sections, which were written from documentation research in 2026-04 and re-verified against current sources and a real Windows machine: - uiautomation 0.25 (a "0.24" requirement will not resolve to it under 0.x semver); windows-capture 2.0.0 - the recorded 1.5.4 was never published - Chromium 138 ships native UIA by default, so the web-wrapper depth-skip is the primary lever and --force-renderer-accessibility is a fallback - Windows 11 22H2+ moved tray overflow to TopLevelWindowForOverflowXamlIsland - windows-11-arm runners have been GA for public repos since 2025-08-07 - Windows 10 1809 remains the API floor, now qualified with its servicing reality (LTSC 2019 / Server 2019 are the serviced targets) - Notepad is two different apps across Server and Win11 client SKUs - CI tables reflect the real test-windows and test-linux lanes from v0.6.0 Adds the no-convenience-deferral rule: scope moves between sub-phases of a phase, never out of it. The only sanctioned deferral is proven impossibility, evidenced by a ledger row, shipping an honest PLATFORM_NOT_SUPPORTED. 2.14 and 3.14 are no longer deferrable stretch sub-phases. |
||
|
Lahfir
|
dde0b98dc0
|
chore: track planning artifacts with secret and privacy scanning (#109)
Some checks failed
CI / Format (push) Has been cancelled
CI / Rust 1.89 MSRV (push) Has been cancelled
CI / Native check (macOS) (push) Has been cancelled
CI / Native check (Linux) (push) Has been cancelled
CI / Native check (Windows) (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Test (Linux) (push) Has been cancelled
CI / Test (Windows) (push) Has been cancelled
CI / FFI Python Smoke (push) Has been cancelled
CI / FFI Header Drift (push) Has been cancelled
CI / FFI Panic Guard (push) Has been cancelled
CI / FFI Stub-Adapter Passthrough (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
Release / Release Please (push) Has been cancelled
Supply Chain / Audit (push) Has been cancelled
Release / Build (aarch64-apple-darwin) (push) Has been cancelled
Release / Build (x86_64-apple-darwin) (push) Has been cancelled
Release / Build FFI (aarch64-apple-darwin) (push) Has been cancelled
Release / Build FFI (x86_64-apple-darwin) (push) Has been cancelled
Release / Build FFI (x86_64-unknown-linux-gnu) (push) Has been cancelled
Release / Build FFI (aarch64-unknown-linux-gnu) (push) Has been cancelled
Release / Build FFI (x86_64-pc-windows-msvc) (push) Has been cancelled
Release / FFI Release Gates (push) Has been cancelled
Release / Publish to GitHub Release (push) Has been cancelled
Release / Publish to npm (push) Has been cancelled
Release / Publish Skills to ClawHub (push) Has been cancelled
Tracks docs/plans and docs/brainstorms, adds .gitleaks.toml with privacy rules, and runs a history-mode gitleaks scan in CI. Docs and tooling only - no releasable change. |
||
|
Lahfir
|
8ad66b8f21
|
refactor!: remove speculative Win32 private-file layer from core, add real Windows/Linux test lanes (#106) | ||
|
Lahfir
|
3f322728b4
|
feat!: implement Playwright-grade foundation contract
Settle the Playwright-grade reliability contract in agent-desktop-core before the Windows/Linux adapters are built, so they inherit it instead of redesigning it. Every command now observes, waits, verifies, and reports honestly instead of firing blindly. Highlights: capability-supertrait split of PlatformAdapter with not_supported() defaults; canonical role/state vocabulary with live `is --property visible`; display enumeration (`list-displays`) and honest `--screen` with scale factor; truthful Automation permission; `native_id` identity spine; window-id-first resolution; serializable `LocatorQuery` with live `find`; default-on auto-wait before every ref action; three-way `hit_test` occlusion gate; `scroll_into_view` in core; core accessible-name precedence; typed `ActionStep` delivery tier; `ProcessState` and `APP_UNRESPONSIVE`; `LaunchOptions`; baseline-diff desktop signals (`wait --event`); typed clipboard (`Text`/`Image`/`FileUrls`); mouse modifier chords and `mouse-wheel`. Hardened through a 35-reviewer pass with independent validation and a green live e2e gate (109/0), plus a head-vs-main performance comparison harness. BREAKING CHANGE: default-on auto-wait changes the timing of every previously-untouched ref-action call (bounded 5000 ms default; `--timeout-ms 0` restores single-shot). `ENVELOPE_VERSION` is now `2.1` (adds the `APP_UNRESPONSIVE` code and process state in error details). FFI ABI major is `3` (append-only struct evolution; `wait --event` is intentionally not exposed over FFI). The legacy string clipboard API is removed in favor of typed content. `key-down`/`key-up` fail closed until daemon-owned held input exists. `close-app` verifies termination and the osascript fallback path is removed. `--text` matching is subtree containment: `find --text X --first` returns the outermost matching container. |
||
|
Lahfir
|
e3e1872ff3
|
feat: add trace viewer and replay artifacts
Adds the trace read/replay layer on the session-first foundation: `trace show` merges per-process segments into one deterministic timeline (bounded JSON for agents), and `trace export` renders a single self-contained, XSS-safe HTML viewer for humans. Opt-in `session start --screenshots` captures pre/post-action screenshots and refmap copies; command.start/end boundary events and a versioned trace.meta header make a step-by-step replay reconstructable. Redaction is hardened so raw caller arguments never leak into trace-reachable error messages. Available across CLI, batch, and FFI. |
||
|
Lahfir
|
35fa914b52
|
feat: make sessions the first-class trace container
Sessions now own tracing: `session start` creates a manifest-gated trace sink with per-process JSONL segments under ~/.agent-desktop/sessions/<id>/trace/, so agents set the session once instead of passing --trace on every command. Bare --session stays snapshot-namespace-only; --trace <path> still overrides for CI and one-offs. Adds session start/end/list/gc, activation resolution (flag > AGENT_DESKTOP_SESSION > pointer), FFI verification tests, and status fields for session_id + tracing. |
||
|
Lahfir
|
9023f331b3
|
feat(ffi): Phase B and C — Python smoke harness, parity gates, build.rs codegen (#77)
Completes the FFI plan: U9 adds a Python ctypes smoke harness proving the C ABI from a non-Rust host; U10 adds cross-platform parity CI gates for header drift, codegen drift, panic-unwind, and stub-adapter passthrough; U11 generates the command-backed wrappers from build.rs templates with a committed drift-gated output. The C ABI stays byte-identical and the surface is Windows and Linux ready with no new FFI code. |
||
|
Lahfir
|
9f144c2caf
|
fix: harden macos stale ref resolution (#62)
* fix: harden macos stale ref resolution * fix: tighten stale ref fallback resolution * fix: fail closed for gone titled windows * refactor: tidy stale ref resolver naming |
||
Lahfir
|
c0d52334c5 | docs: update phase roadmap for v0.3.0 | ||
|
Lahfir
|
0c0a5b8dbc
|
fix: harden agent-desktop reliability (#54)
* feat: add strict ref reliability core
* feat: add session-scoped reliability diagnostics
* fix: skip finder pseudo windows for snapshots
* fix: report wait and trace failure context
* fix: harden ref action reliability
* fix: close reliability review findings
* fix: harden reliability edge cases
* fix: close final reliability edge cases
* fix: address reliability follow-ups
* docs: update reliability docs and skills
* fix: harden wait and ref action reliability
* refactor: centralize reliability helpers
* fix: stabilize macos ref resolution
* refactor: organize binary crate modules
* fix: harden ref action reliability
* fix: harden ref action reliability
* docs: compound reliability patterns
* fix: harden ref reliability edge cases
* fix: harden source-window ref resolution
* fix: preserve safe window title fallback
* fix: make explicit snapshots session-independent
* fix: harden ref fallback resolution
* fix: fail closed on uncertain ref fallback
* chore: strip inline comments and enforce docstrings
Replace inline // comments with /// docstrings where they carry non-obvious
contract, and add a pre-commit guard so inline comments cannot regress.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(review): preserve action results, delegate timeout resolution, tighten core boundaries
Apply verified code-review fixes: ref-action release failures no longer
mask successful action results (prevents double-dispatch on retry);
resolve_element_strict_with_timeout defaults to delegating so strict-only
adapters support wait --element; wait --text reports count only when
--count is requested; latest-refmap refresh logs load failures instead of
silently serving stale refs; InteractionPolicy moved to its own module and
actionability/trace modules scoped pub(crate) per file rules; duplicate
wait test helper extracted to shared support module; timeout error
constructors deduplicated; redaction test covers description; policy
focus-denial path covered; skills document steps array, actionability
details, trace redaction, and batch trace inheritance.
* fix(review): eliminate per-element AX round trips and close remaining review findings
Fold AXPosition/AXSize and the scrollbar probe into the existing
AXUIElementCopyMultipleAttributeValues batch so tree traversal and the
actionability preflight pay one IPC per element instead of up to four;
A/B benchmark shows strictly-faster snapshots with identical ref counts
and scroll capabilities (Finder 4.5s -> 2.4s same-session, Docker
Desktop parity at 440 refs / 933 scroll-capable nodes).
Consolidate the CLI and FFI ref-action pipelines into one core
execute_resolved path (actionability, tracing, and dispatch semantics
live once; FFI passes a default context). Remove the no-context
execute() shims from is/right-click/snapshot/wait and the test-only
helper shims; every command now takes an explicit CommandContext.
Split the macOS resolver into resolve (orchestration), resolve_search
(candidate collection), and resolve_classify (strict classification),
clearing the 400-LOC ceiling with room to grow.
Notification waits now retry transient baseline failures inside the
timeout budget with the same retryable gate as window/text waits and
report last_error in timeout details instead of aborting on the first
flake; a baseline is never fabricated.
Refmap writes clean up their temp file on failure, stale *.tmp orphans
are swept under the store lock, and save_existing_snapshot re-verifies
snapshot ownership inside the owning store's write lock with bounded
re-discovery before deterministically recreating in the caller's store.
Coverage hardening: zero-budget wait timeout shape, wait --text
--count 0 absence detection, ref-action pipeline call-count guard
(1 resolve / 1 live read / 1 dispatch), duplicate snapshot-id collision
on load, pruned-everywhere recreation, tmp sweep and rename-failure
cleanup, FFI AMBIGUOUS_TARGET last-error code assertion.
* docs: document notification retry, error-code contract, and diagnostics sensitivity
Note transient-error retry and last_error timeout detail on wait
--notification; make explicit that agents branch on error.code (message
and suggestion text is informational); warn FFI consumers that
ad_last_error_details may carry on-screen element names, values, and
window titles and should stay out of shared log surfaces.
* fix: validate find roles against the canonical vocabulary
find --role with a role no adapter can emit (textarea, typos) silently
returned ok with zero matches, reading as 'element absent' when the
query could never match. Role queries now resolve through a canonical
vocabulary in core: common text-input aliases (textarea, textbox,
searchfield) normalize to textfield case-insensitively, and unknown
roles fail with INVALID_ARGS carrying details.valid_roles so agents can
self-correct.
The macOS role mapping becomes a sorted single-source table with
binary-search lookup, and a conformance test pins every emitted role
(plus the unknown fallback and the synthesized cell role) to core's
CANONICAL_ROLES — the cross-platform contract Windows/Linux adapters
must map their native vocabularies into, enforced by the same
table + test pattern rather than convention.
* refactor: derive find role hints from the live tree, drop hardcoded vocabulary
Replace the canonical-role allow-list (and its hard INVALID_ARGS
rejection) with a tree-derived approach. A role filter that matches
nothing now returns ok with roles_present — the distinct roles actually
in the searched tree — so the caller distinguishes 'none on screen' from
a wrong role name and self-corrects. This needs no central role list: a
role any adapter newly emits surfaces in roles_present automatically,
with nothing to keep in sync across core and the platform crates.
A tiny role-query normalizer keeps the ergonomic win (textarea, textbox,
searchfield fold to textfield, case-insensitive) but never gates or
rejects — it is a synonym shim, not a vocabulary. The macOS role table
returns to its plain match form; the cross-crate canonical-vocabulary
list and its conformance test are gone.
* refactor: move protected-process knowledge out of core into the adapter
close-app hardcoded macOS/Unix process names (loginwindow, windowserver,
dock, launchd, finder) inside core, baking platform-specific knowledge
into the platform-agnostic crate. Windows would need csrss.exe/
winlogon.exe, Linux gnome-shell/Xorg. Add PlatformAdapter::
is_protected_process (default denies nothing); the macOS adapter owns its
list with substring matching over display and bundle identifiers. core's
close-app just asks the adapter. Also genericize a macOS-flavored test
fixture string so core carries zero native vocabulary even in tests.
Verified: core has no platform-native references in non-test source, no
cfg(target_os) gates; the only remaining cfg(unix)/libc use is securing
core's own refmap/trace/lock files with non-unix fallbacks.
* fix: stop close-app claiming a graceful quit it cannot confirm
close-app returned closed:true the instant a graceful quit was *sent*,
while the app was still running behind an unsaved-changes dialog —
a false completion claim. Empirically (NSWorkspace.runningApplications):
a clean quit completes in ~0.2s, a dialog-blocked quit never completes
on its own, and macOS confirms only that the quit request was sent, not
that the app terminated. Verifying by polling would add seconds of
latency on the exact (blocked) case it is meant to catch, so we do not
poll.
Graceful close now reports { method: graceful, requested: true } —
truthful and instant, no closed claim. --force is a synchronous SIGKILL,
so it reports { method: force, requested: true, closed: true }. Callers
needing graceful confirmation observe via list-apps / wait --window and
can drive a save dialog with snapshot + find, which is the agent-native
path.
* feat: add drag --drop-delay for reliable macOS drop registration
macOS drop targets need the dragged item to dwell over them before they
register as the destination; too short and the gesture lands as a drag
with no drop. The dwell was a hardcoded 500ms dead sleep. Expose it as
--drop-delay <ms> (CLI), drop_delay_ms (DragParams/AdDragParams, 0 =
adapter default sentinel matching duration_ms), and replace the dead
sleep with an event-driven dwell that posts LeftMouseDragged over the
destination every 16ms so the target stays highlighted instead of
dropping the drag mid-pause.
DRY: the C-to-core drag conversion (duration/drop-delay zero-sentinel)
was copied across three FFI sites; collapse them into AdDragParams::
to_core(). FFI ABI: AdDragParams gains drop_delay_ms (header + repr +
header-compile test). Tests: core threads the value into params and
response and omits the field when unset; FFI maps both optionals.
* fix: make action-bearing elements ref-able so scroll/expand can target them
E2E testing against a diverse fixture app surfaced that disclosure
(Expand/Collapse/Click) and scrollarea (Scroll) advertise actions but
never received refs — they are not in INTERACTIVE_ROLES — so the scroll,
expand, and collapse commands required a <REF> their own target roles
could never have. The commands were uninvokable against their primary
targets.
Ref allocation now gates on addressability, not role alone: an element
is ref-able if its role is interactive OR it advertises a primary action
(any action other than a bare SetFocus, which would ref-allocate inert
focusable containers). scrollarea and disclosure become ref-able;
scroll now works against a real app. Ref-count impact is modest
(fixture 61->72, Finder ~262).
Tests assert action-bearing containers get refs, SetFocus-only and inert
elements do not, and interactive roles stay ref-able without actions.
Contract docs (CLAUDE.md, SKILL.md) updated.
* fix: eliminate vacuous AX successes and harden resolution
Dogfooding the binary against a real fixture app surfaced five cases where a
command reported success without producing the effect, or failed with the wrong
error. Each is verified by independent before/after observation in the E2E
harness.
- is_menu_open no longer treats a latent AXMenuBar as an open menu, so select
and wait --menu-closed stop seeing a permanently-open menu.
- set-value coerces the written AXValue to the element's existing CFNumber/
CFBoolean/CFString type and verifies numerically, fixing sliders; steppers
converge via AXIncrement/AXDecrement when AXValue writes are vacuous.
- double-click only claims success when the element advertises AXOpen; otherwise
it fails closed instead of reporting a non-existent double-click.
- a completed resolution pass that proves a ref absent downgrades a deadline
TIMEOUT to STALE_REF so removed elements fail with the correct code.
- expand/collapse verify the disclosure state and fall back to a press-toggle
for press-driven disclosures; press-toggled containers expose EXPAND/COLLAPSE.
roles.rs adds the disclosure expandable role and normalizes textarea/textbox/
searchfield role queries to textfield.
* feat: add Playwright-style headed/headless interaction mode
Ref actions now run in exactly two modes. Headless is the default: semantic
accessibility operations only, no cursor movement, and a fail-closed
POLICY_DENIED when only a physical gesture would work. The global --headed flag
upgrades every ref action to permit focus stealing and cursor movement, so the
chain's physical click/double-click/scroll/keypress fallbacks can complete. The
AX path is always tried first, so --headed never regresses headless-capable
elements; it only adds fallbacks for elements that need a real gesture.
- CommandContext::request(action, base) builds the per-command request: each
command declares its headless base (pure-AX headless; type uses focus_fallback
because typing requires focus but never moves the cursor) and --headed upgrades
any base to the headed policy.
- The internal/FFI "physical" policy is renamed "headed" throughout, including
the C ABI enum (AD_POLICY_KIND_HEADED keeps discriminant 2) and bindings.
- Raw-input commands (press, hover, drag, mouse-*, key-down/up) are unchanged:
always physical, mode-independent low-level escape hatch.
- Unit tests assert every ref command is headless by default and headed under
--headed; docs (CONCEPTS, CLAUDE, skills) describe the two-mode contract.
* test: add E2E fixture app and dual-mode harness
Drives the release binary against a real SwiftUI/AppKit fixture and verifies
every effect by independent before/after observation — never the command's own
ok:true — so a command that reports success without an effect is caught. This is
the layer mock-adapter unit tests cannot cover: it exercises the contract
against the real macOS Accessibility API.
- AgentDeskFixture.swift exposes a fixed, diverse AX surface (native AppKit
slider/stepper, gesture-only and ambiguous controls, a sheet, a press-toggled
disclosure, async-appearing elements, a drag canvas). It is never tuned to
make a command pass; a failure is a finding about the CLI or the harness.
- run.sh drives every ref-action command in BOTH headless and --headed mode with
mode-specific target values, plus the double-click discriminator (headless
fails closed with POLICY_DENIED, --headed completes) that proves the two modes
differ. It also covers strict resolution, wait predicates, skeleton drill-down,
sessions, trace redaction, surfaces, drag, expand, and force-close.
- The compiled fixture .app is a build artifact (gitignored; built on demand).
Run: cargo build --release && bash tests/e2e/run.sh (needs AX permission).
* fix: close review runtime, correctness, and security gaps
Addresses validated findings from the branch code review:
- chain: thread the chain deadline into increment_to_value so a non-converging
stepper cannot spin up to 1024 AX round trips and blow past the timeout.
- mouse: a RAII guard posts LeftMouseUp if any fallible step of a drag returns
early, so an error can never leave the mouse button held down system-wide.
- ffi: cap caller-supplied state/action/path counts before from_raw_parts,
mirroring the existing MAX_MODIFIERS_PER_COMBO guard, to reject out-of-bounds
reads from a garbage C count.
- wait: the actionable predicate now forwards the structured ActionabilityReport
from error.details instead of dropping it to a flat message, so agents can see
which check is blocking.
- scroll: gate the row-select fallback on policy.allow_focus_steal so a headless
scroll can no longer silently change the user's table selection.
- actionability: delete the unreachable stable->StaleRef branch (stability_check
never fails by design) and its now-dead failed_check helper.
- status: delete two pub wrappers with no production callers; the test now drives
the real execute_with_report_with_context entry point.
* refactor: extract disclosure chain steps under the 400-LOC limit
chain_steps.rs had grown to 402 lines, over the hard per-file limit. The six
press-toggle disclosure helpers form a cohesive group and move cleanly into a
sibling chain_disclosure_steps.rs (following the chain_web_steps/
chain_menu_steps pattern); chain_defs.rs references the new module. No behavior
change.
* feat: focus the target window before ref-addressed physical input
Ref-action physical fallbacks (click/scroll/type) already brought the target
app frontmost before synthesizing CGEvents. The raw-input commands (drag, hover,
mouse-*) resolving a point from a ref did not — the resolver had the pid but
discarded it, so the adapter saw only coordinates and synthetic events could
land on whatever window happened to be frontmost.
Add a best-effort focus_app(pid) to PlatformAdapter (macOS uses
ensure_app_focused; other adapters default to not_supported). The point resolver
now focuses the ref's app before returning, so every physical interaction that
targets a known element raises its window first. Coordinate (--xy) input is
unchanged: the caller owns the target there.
* fix: add AdDragParams size guard and document the ABI breaks
AdDragParams gained a drop_delay_ms field but, unlike AdRefEntry, had no size
guard — an old caller's smaller allocation would let Rust read past it and turn
stack garbage into a real drop delay. Add AD_DRAG_PARAMS_SIZE, ad_drag_params_size(),
a compile-time layout assertion, and a zero-init note, matching the ref-entry
pattern.
This branch makes several consumer-visible contract changes that release
tooling must cut as a major. They are gathered here because the release workflow
ships the C header as an artifact.
BREAKING CHANGE: the C ABI and CLI/JSON contract changed on this branch.
- AdPolicyKind: AD_POLICY_KIND_PHYSICAL is renamed AD_POLICY_KIND_HEADED
(discriminant 2 unchanged, so compiled binaries are safe; source-level C
consumers must rename). No back-compat alias is kept — "physical" is gone.
- AdRefEntry grew (caller-allocated input); validate layout with
AD_REF_ENTRY_SIZE / ad_ref_entry_size().
- AdDragParams grew; validate with AD_DRAG_PARAMS_SIZE / ad_drag_params_size()
and zero-initialize before use.
- close-app graceful response no longer includes closed:true; it returns
{ method: "graceful", requested: true } because a graceful quit cannot be
synchronously confirmed.
* test: harden and expand the E2E proof layer
Closes the honesty gaps the review found and covers the interactions that were
missing, all verified by independent before/after observation:
- twins: the fixture twins now record distinct effects (twin-a/twin-b) and the
assert requires the ADDRESSED twin to fire (or AMBIGUOUS_TARGET), instead of
passing on any ok:true.
- click: click-status is a counter, so the headed pass must observe a fresh
increment rather than inheriting the headless pass's value.
- adds triple-click + hover (headed gestures), tab selection (TabView tabs are
radiobuttons), context-menu open + item selection, and menu-bar enumeration
via --surface menubar.
- adds a performance section reporting per-command CLI wall-clock (snapshot,
find, get, click, set-value, type) with a soft <2s snapshot gate.
- documents the SwiftUI CommandMenu and cross-app drop limitations as tracked
notes, not silent skips.
* docs: trim CLAUDE.md to standards and document gesture headless-capability
- CLAUDE.md: remove ~130 lines of reference material that duplicated code,
Cargo.toml, or the skills (full PlatformAdapter trait dump, Key Types listing,
macOS API listings, dependency/build-config tables, the 54-command table).
Replaced the stale trait dump with a pointer to adapter.rs (which also fixes
the review's stale-trait-docs finding) and kept only the non-obvious gotchas.
CLAUDE.md is now standards, invariants, and conventions.
- Document, on macOS (Phase 1), which gestures have a headless path: most ref
actions do; double-click via AXOpen; triple-click/hover/drag are cursor
gestures with no AX equivalent (physical only). The command surface is
platform-agnostic — a future Windows/Linux adapter that exposes a headless
path lights it up with no command or core change. Added to README and the
interaction reference.
- FFI skill: AD_POLICY_KIND_PHYSICAL is now AD_POLICY_KIND_HEADED.
* docs: capture gesture headless-capability learning and refresh policy docs
- Add best-practices/macos-gesture-headless-capability: which desktop gestures
have a headless AX path on macOS (double-click via AXOpen; triple-click/hover/
drag are physical-only; SwiftUI controls vs native AppKit), and why the command
never decides — the platform adapter owns headless-vs-physical.
- Refresh two policy learnings for the physical->headed rename: ActionRequest::
physical -> headed and AD_POLICY_KIND_PHYSICAL -> AD_POLICY_KIND_HEADED, noting
the new global --headed upgrade path via CommandContext::request.
* fix: address review P2 correctness and reliability findings
- wait: cap each ref-resolution attempt (750ms) so a slow resolve cannot
consume the whole wait budget on the first poll; the predicate is re-checked
across the full timeout.
- wait: make LatestRefCache timing fields private (no external readers).
- wait: add a unit test for --menu-closed (asserts it waits for open=false).
- close-app: make the graceful and force responses symmetric — both carry
`closed` (force confirms true; graceful cannot confirm, so false) instead of
graceful silently omitting the field.
- close-app: test adapter-error propagation.
- adapter: the default resolve_element_strict_with_timeout now logs that it does
not enforce the deadline, so an adapter that forgets to override it is visible
in traces.
- type: a RAII guard restores the user's clipboard on every scope exit (success,
error, panic) during the paste-based non-ASCII path, shrinking the clobber
window to an unpreventable SIGKILL.
Note: the reviewer's "fail fast on AMBIGUOUS_TARGET with a pinned snapshot"
suggestion is not applied — existing tests prove transient ambiguity resolves on
retry even with a pinned ref (ambiguity is a property of the live tree, not the
refmap), so failing fast would regress intentional, tested behavior.
* perf: trim actionability preflight and resolve-search allocations
- action_list: gate the AXValue and AXExpanded `is_settable` probes on whether
the role could plausibly carry that capability (unknown roles always probe),
skipping up to two AX round trips per preflight on common click-only targets.
No capability is lost — value/expandable roles still probe.
- resolve_search: reuse one scratch FxHashSet across nodes instead of allocating
a fresh dedup set per node during path and recursive search.
* refactor: split actionability types into one file each
ActionabilityStatus, ActionabilityCheck, and ActionabilityReport move into their
own files under a new actionability/ module (following the tree/ and actions/
folder pattern); mod.rs keeps the check logic and re-exports the types. Honors
the one-domain-type-per-file rule without changing behavior.
* test: split fixture under the LOC limit and fix swift-ios issues
- Extract the reusable fixture components (status readout, native AppKit
slider/stepper, drag canvas, card) into FixtureComponents.swift so each file
is under the 400-LOC limit; build.sh now compiles every .swift in the dir.
- NativeSlider/NativeStepper implement updateNSView so a SwiftUI binding change
syncs back to the NSView.
- The fixture no longer steals focus unconditionally on launch
(activate ignoringOtherApps:false), so it cannot mask headless-policy focus
violations; the harness drives focus explicitly.
- build.sh pins the SDK and a macOS 13 deployment target for reproducible builds.
* docs: document the optional error.details field and roles_present shape
- SKILL.md: note that the error object may carry an optional `details` (the
actionability report, AMBIGUOUS_TARGET candidates, or a wait TIMEOUT's last
observed state) and that responses should be parsed leniently — `details` and
future fields are additive.
- commands-observation: show the no-match `find` response with the
`roles_present` hint so callers can tell a wrong role name from "none on
screen".
* fix: gate the ref-gesture focus raise on the interaction policy
Ref-addressed hover/drag raised the target app unconditionally, violating
the headless no-implicit-focus-steal contract. The point resolver now
returns the owning pid instead of focusing, and commands decide: headless
never raises, --headed raises once (drag focuses only the from-app, fixing
the cross-app double-focus). Responses report focused:true so multi-app
agents can detect the frontmost change.
* fix: abort failed drags at the origin and disarm the guard only on success
The mouse-up guard disarmed before the final fallible up-event, so a
failed final post left the button held. Worse, its corrective release
fired at the unreached destination, silently committing an aborted drag
as a completed drop (CGEvents resolve at their embedded coordinates).
The guard now owns the release: it disarms only after the up actually
posts, and an early return cancels by dragging back to the origin and
releasing there.
* fix: enforce the chain deadline inside increment steps
All dispatch sites construct ChainContext with deadline: None, so the
remediation parameter on increment_to_value never received a value and
the 1024-iteration loop ran unbounded by the chain timeout. The chain now
pins its resolved deadline into the context every step observes. Also
extracts the pure write-verification predicates and their tests to
chain_verify.rs, bringing chain.rs back under the 400-LOC limit.
* fix: pin the AdAction ABI layout and bound FFI string and array inputs
AdDragParams is embedded by value in AdAction, so its 8-byte growth grew
the struct C callers pass to ad_execute_action with no size guard —
old-layout callers under-allocate and stack garbage becomes a live
drop_delay_ms. Adds AD_ACTION_SIZE / ad_action_size() with a layout pin,
matching the AdRefEntry pattern.
Also hardens the input boundary: C strings are decoded with a bounded
NUL scan (AD_MAX_STRING_BYTES, sized for CLI argv parity) so a missing
terminator cannot walk arbitrary memory, and the single coarse 1024
array cap becomes published per-field caps (AD_MAX_REF_STATES/ACTIONS/
PATH_DEPTH) with tests just over each limit.
* perf: replace fixed input settle sleeps with state polls
ensure_app_focused slept 50ms per physical input even when the app was
already frontmost; it now polls AXFrontmost (1ms, 50ms deadline).
disclosure_settled slept an unconditional 40ms up to three times per
expand/collapse; it now polls the disclosed state (5ms, 200ms deadline),
converging immediately on fast UIs. The kAXFocusedAttribute settability
probe is gated by role_may_accept_focus, mirroring role_may_bear_value,
and key dispatch reuses ensure_app_focused instead of its inline
duplicate.
* feat: check a specific action in wait --predicate actionable
The actionable predicate hardcoded Click, so wait-then-type flows got a
false ready on fields that cannot accept text (the editability check only
runs for editing actions). --action selects click (default), type,
set-value, or clear, and the preflight mirrors each command's real base
policy (type uses its focus-fallback base).
* fix: redact title, url, help, and placeholder keys in traces
Window titles, URLs, tooltips, and placeholder text carry user content
just like names and values; the redaction list now covers them.
* test: harden the e2e harness and unify fixture AX labels
The harness now fails setup loudly when the fixture build or AX trust is
missing, rebuilds the fixture when sources are newer than the bundle,
asserts hover only against a freshly observed state, and force-collapses
the disclosure so the expand test proves a real flip. The slider/stepper
labels live solely on the NSViews (the AX-actionable elements), removing
the macOS-version-dependent race between two label sources, and the drag
canvas reports a zero frame when detached from a window.
* docs: record the perf commit type, pre-1.0 bump policy, and error details field
Adds perf: to the allowed commit types (release-please already maps it to
a Performance changelog section), records the pre-1.0 versioning policy so
a BREAKING footer is expected to cut a minor rather than a major, and
shows the optional error.details object in the error envelope docs.
The gitignored local AGENTS.md mirror got the same contract sync.
* fix: surface increment deadline expiry as timeout with the observed value
A chain deadline firing mid-increment returned Ok(false), so the step was
recorded as skipped, the chain exhausted into ACTION_FAILED, and the
control sat at a half-applied value the caller could not see — post-state
is only read on success, and ACTION_FAILED recovery guidance points away
from retrying. Expiry is now a TIMEOUT error carrying value_before,
value_at_timeout, target, and a mutated flag in details.
* perf: cap the disclosure settle poll to the chain deadline and widen its interval
The settle poll could spend 3 x 200ms x 5ms-interval reads (~360 IPCs)
per expand/collapse and overshoot the chain's own deadline. A new
CustomWithDeadline chain step threads the chain deadline into the
disclosure steps, the settle budget is min(200ms, remaining chain
budget), and the interval widens to 20ms (~30 IPCs worst case).
* fix: enforce the protected-process guard inside the adapter close path
The guard lived only in the CLI command layer, so ad_close_app could
force-kill session-critical processes (loginwindow, WindowServer, Dock)
that the CLI refuses. close_app_impl now refuses them before any side
effect with the exact CLI error contract, making CLI, FFI, and any future
consumer behave identically; the CLI preflight remains as an earlier
check against the same predicate.
* fix: make focused semantics honest and confirm window focus by polling
ensure_app_focused set AXFrontmost unconditionally and reported success
identically whether or not a raise happened; it now no-ops when the app
is already frontmost, so Ok (and the focused:true response field) means
"frontmost ensured" exactly as documented. focus_window_impl gains the
same confirmation poll after its raise, and the poll interval widens to
5ms (10 reads max in the 50ms window).
* fix: attach abort-state guidance to drag failures and document cancel limits
Drag synthesis errors surfaced as bare INTERNAL with no hint about the
gesture's end state. Failures now carry a suggestion stating the button
was released back at the origin (best-effort), no drop was committed,
and where the cursor ends; the guard doc spells out the two best-effort
limits (corrective posts can fail; a self-drop at the origin is a no-op
for most targets).
* refactor: split oversized files by responsibility under the 400-LOC limit
helpers_tests (426) splits into resolution/window/pipeline tests, a
ref-action+trace test file, and a shared entry-builder support module.
wait_element_tests (414) splits into predicate-behavior tests and
resolution/lifecycle tests over a widened wait_test_support. wait.rs
(395) loses the element-wait loop to wait_element.rs, and refs_store
(397) moves its tmp-cleanup/retention methods to a refs_store_prune
child module (declared via #[path] so the split keeps base_dir and
snapshots_dir private to the store).
* refactor: build the actionable preflight request per action name at parse
The policy mirror lived in a separate helper with a catch-all arm, so a
future action name could silently inherit the headless policy. Parse now
maps every --action name to the exact ActionRequest its real command
runs (type is the only focus-fallback), the catch-all is gone, and a
test pins each name's policy.
* refactor: move point resolution and the focus helper to point_resolve
PointResolveArgs, ResolvedPoint, the ref-or-xy resolver, and
focus_for_physical_input were accumulating in helpers.rs alongside
unrelated ref-action plumbing; they now live in a dedicated
point_resolve module consumed by hover and drag.
* test: keep the drag canvas AX label on the NSView only
The DragCanvas carried two label sources (the NSView and a SwiftUI
modifier on its representable), the same macOS-version-dependent race
the slider/stepper fix removed; the harness-facing label now lives
solely on the AX-actionable NSView.
* docs: sync skills and header with the focused, redaction, and wait contracts
Documents the ensured (best-effort, already-frontmost-aware) semantics of
focused:true and its absence-vs-false meaning, the four redaction keys
added in round 2 plus the substring-match behavior, the INTERNAL error
recovery row, the FFI wait-surface asymmetry, the cross-app drag
occlusion caveat, and the AdDragParams/AdAction layout history with the
adjudicated pre-1.0 breaks so fresh reviews stop re-finding them.
* fix: surface settle-wait deadline truncation as timeout with a schema discriminant
A chain deadline truncating the disclosure settle wait returned a plain
step failure, exhausting into ACTION_FAILED — the same masking class
fixed for increments — even though the triggering action may still land
after the truncated wait. Settle exits are now classified: full-budget
misses stay step failures, deadline-truncated waits raise TIMEOUT with
the wanted/observed state, and the poll sleeps are clamped so a tight
deadline still gets at least one read. All TIMEOUT details now carry a
kind discriminant (wait_timeout vs chain_deadline) so agents can branch
without sniffing field names.
* fix: match protected processes exactly, not by substring
'docker'.contains('dock') permanently blocked close-app for Docker,
FinderSync-class apps, and anything else embedding a protected name.
Matching is now an exact lowercase name or an exact dot-separated
bundle-id component, so Dock and com.apple.dock stay protected while
Docker, Docker Desktop, FinderSync, and PathFinder stay closable —
pinned by false-positive tests.
* fix: raise the element's window before the physical click fallback
CGEvents land on the topmost window at the click point, so an app being
frontmost is not enough when the target element lives in a background
window of that app — the physical fallback clicked whatever overlapped
it, and the skip-raise-when-frontmost optimization widened the window
for that. click_via_bounds now raises the element's own AXWindow (AXRaise,
AXMain fallback, brief confirmation poll) via a shared window_ops helper
that focus_window_impl reuses. Verified live: a headed click on a ref in
an occluded Finder window raises that window and lands the click in it.
* test: guard ref-action policy coverage against silent gaps
A new ref-action command could ship without a base-policy assertion. A
guard test now scans crates/core/src/commands/ for files calling
context.request( and fails unless each stem appears in the
POLICY_TESTED_COMMANDS list backing the policy assertions.
* fix: carry the protected-process suggestion on the CLI preflight
The CLI-layer guard returned bare INVALID_ARGS while the adapter layer
carried recovery guidance, so agents on the primary surface got 'check
command syntax' for a permanently-disallowed operation and looped on
argument fixes. Both layers now state the same suggestion.
* docs: document the TIMEOUT schemas, chain deadline knob, and roles_present scope
Names the two TIMEOUT details schemas by their kind discriminant with
the mutated-flag retry rule, points chain-deadline recovery at
AGENT_DESKTOP_CHAIN_TIMEOUT_MS instead of --timeout, extends the
roles_present hint to all non-count selection-mode misses, and aligns
the STALE_REF recovery row with the richer error.rs suggestion.
* test: extract the scroll card and document drag-canvas data flow
AgentDeskFixture.swift sat at exactly 400 lines; the scroll card is
fully self-contained (its offset state never leaves the card), so it
moves to FixtureCards.swift as a standalone view with zero bindings,
landing the main file at 374. Harness-facing labels are byte-identical.
DragCanvas gains two intent comments distinguishing the deliberately
empty updateNSView from a forgotten sync.
* chore: pin ABI sizes for C consumers and explain the prune module split
C11-gated _Static_asserts mirror the Rust-side layout pins so a C
consumer compiling against a drifted header fails at build time, and
the production #[path] prune module carries its privacy rationale.
* test: give racing wait tests a deterministic budget
Three wait tests used a 1ms timeout that can elapse before the loop's
first resolution attempt under load, flaking on machine pressure (the
ambiguous-resolution test needs at least one attempt to record its
observation). 50ms guarantees the first attempt without slowing the
suite.
* docs: capture three round-4 review learnings and grow the concept map
Documents the abort-state contract for multi-step physical input (guard
disarm ordering, origin release, end-state suggestions), the three-layer
repr(C) size-pinning discipline born from the AdAction silent-growth
incident, and the named-arms-plus-exhaustiveness-guard pattern for
policy/dispatch mirrors. CONCEPTS.md gains Action Chain and Protected
Process and refreshes Coordinate Fallback with the window-topmost rule.
* docs: refresh six learnings against the enhanced-reliability branch
Brings the learning corpus back in line with code that moved this
branch: the gesture-capability and policy docs now describe the
window-level raise in the physical path and link the new abort-state
doc, the reliability contract documents the TIMEOUT kind discriminant
and the wait --action per-name policy variant, the FFI review rule
covers structural repr(C) size drift alongside behavioral parity, the
allocator doc records how the config struct absorbed four more fields
in one place, and the fingerprint doc names the real tri-state decode
error type. Three docs verified accurate with no edits.
* docs: sync roadmap with current reliability contracts
* fix: harden desktop action reliability
* fix: harden reliability review regressions
* fix: close final reliability review gaps
* fix: close reliability review gaps
* test: avoid raw pointer mutation in ffi free tests
* refactor: trim reliability branch dead code
* fix: close reliability review gaps
BREAKING CHANGE: the C ABI AdActionResult layout now includes action steps; C consumers must rebuild against the updated agent_desktop.h header.
* fix: close reliability review gaps
* ci: scope cache hash inputs
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Lahfir
|
a520b7b574 |
docs: scrub public phase roadmap
Some checks failed
CI / Format (push) Has been cancelled
CI / Test (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
Release / Release Please (push) Has been cancelled
Supply Chain / Audit (push) Has been cancelled
Release / Build (aarch64-apple-darwin) (push) Has been cancelled
Release / Build (x86_64-apple-darwin) (push) Has been cancelled
Release / Build FFI (aarch64-apple-darwin) (push) Has been cancelled
Release / Build FFI (x86_64-apple-darwin) (push) Has been cancelled
Release / Build FFI (x86_64-unknown-linux-gnu) (push) Has been cancelled
Release / Build FFI (aarch64-unknown-linux-gnu) (push) Has been cancelled
Release / Build FFI (x86_64-pc-windows-msvc) (push) Has been cancelled
Release / Publish to GitHub Release (push) Has been cancelled
Release / Publish to npm (push) Has been cancelled
Release / Publish Skills to ClawHub (push) Has been cancelled
|
||
|
Lahfir
|
c16a91e9bd | docs: keep phase planning artifacts local | ||
|
Lahfir
|
9435b3c193 | docs: harden phase 2 windows plan | ||
|
Lahfir
|
1fb5a7d51e
|
fix: stabilize empty accessibility identity refs
* fix: treat empty accessibility text as missing identity * fix: align help and skill snapshot docs * fix: preserve accessibility descriptions in refs * fix: resolve refs after window title drift * fix: harden stale ref identity resolution * ci: add explicit CodeQL workflow * fix: fail closed when scoped refs drift |
||
|
Lahfir
|
1291a9cdbf
|
refactor!: unify command execution contracts
Unify CLI and batch dispatch around the typed command path, centralize command policy and ref resolution, harden macOS action verification, split command tests from implementation, and add package/release guardrails. BREAKING CHANGE: CLI and batch execution now share the typed command path and current command argument contracts. BREAKING CHANGE: Ref-consuming commands use snapshot-scoped refs; deterministic consumers should pass snapshot_id and handle SNAPSHOT_NOT_FOUND. BREAKING CHANGE: permissions and status now return PermissionReport fields for accessibility, screen_recording, and automation instead of a single boolean status. BREAKING CHANGE: PermissionState gains NotRequired; macOS automation now reports not_required instead of unknown. BREAKING CHANGE: right-click now separates action success from menu verification; consumers should inspect menu or menu_probe instead of assuming every right-click returns an inline menu. BREAKING CHANGE: focus-window now confirms OS focus and returns ACTION_FAILED when focus does not settle; data.focused.is_focused is true on success. BREAKING CHANGE: PlatformAdapter::execute_action now takes ActionRequest, and permission probing uses permission_report/request_permissions. BREAKING CHANGE: FFI ad_execute_action now defaults to headless policy. Consumers that need focus fallback or cursor-moving behavior must call ad_execute_action_with_policy with AD_POLICY_KIND_FOCUS_FALLBACK or AD_POLICY_KIND_PHYSICAL. BREAKING CHANGE: FFI ad_check_permissions no longer treats unknown accessibility permission as success; stub-style unknown probes return ERR_PLATFORM_NOT_SUPPORTED and macOS ambiguous unknown returns ERR_INTERNAL with last-error detail. BREAKING CHANGE: JSON response envelopes now report version 2.0; parsers pinned to 1.0 must branch or update. BREAKING CHANGE: focus now uses accessibility focus without cursor movement; callers that need physical focus must use explicit mouse or physical-policy paths. BREAKING CHANGE: chain execution deadlines now return TIMEOUT instead of ACTION_FAILED when the target app does not respond before the chain deadline. |
||
|
Lahfir
|
99fd6799ec |
docs(phases): expand Phase 2 plan invariants and dependency table
Distributes Phase 2 plan research findings into roadmap: headless-first invariant, Windows engineering invariants, DeliverFiles rationale, build.rs registry decision, and updated dependency pins. |
||
|
Lahfir
|
2d373708e7
|
docs(phases): sync roadmap + distribute gap findings + codify DRY invariant (#28)
Docs-only sync of `docs/phases.md` against the v0.1.13 shipped reality, with four research agents' findings distributed into Phase 2-5 as concrete objectives and the command-surface DRY invariant codified at the top of the document. |
||
|
Lahfir
|
3cffbd67f6
|
feat(ffi): ship C-ABI cdylib with review hardening and release pipeline (#26)
Lands the full agent-desktop FFI layer with every PR #22 review finding resolved, modular refactor applied, todo-resolve batches (011 + 006) closed, and a production release pipeline that bundles the prebuilt cdylib for 5 target triples alongside the CLI on every GitHub Release. **FFI surface** - Panic-unwind cdylib boundary via `trap_panic` / `trap_panic_ptr` / `trap_panic_const_ptr` / `trap_panic_void` - Runtime main-thread enforcement on every macOS-sensitive entrypoint; TLS errno-style last-error lifetime - Every `#[repr(i32)]` field validated at the C boundary via `try_from_c_enum!` — arbitrary bit patterns return `ErrInvalidArgs` without UB - BFS flat-tree layout with `child_start` / `child_count`; iterative traversal - Opaque list handles (`AdAppList`, `AdWindowList`, `AdSurfaceList`, `AdNotificationList`); opaque `AdImageBuffer` with `_data` / `_size` / `_width` / `_height` / `_format` accessors - `AdNativeHandle` single-owner single-thread contract; zero-on-free makes double-call deterministic - Fail-closed UTF-8 for optional filter pointers (`try_c_to_string` tri-state) - `AdTreeOptions` fully honored in `ad_get_tree` (include_bounds / interactive_only / compact) - Verified notification action identity (`NotificationIdentity` fingerprint) — refuses to press if NC reordered between list and act **Release pipeline** - New `build-ffi` matrix in `.github/workflows/release.yml` producing `libagent_desktop_ffi.{dylib,so,dll}` tarballs for aarch64/x86_64-apple-darwin, x86_64/aarch64-unknown-linux-gnu, x86_64-pc-windows-msvc - macOS `install_name = @rpath/libagent_desktop_ffi.dylib` baked in by `build.rs` and CI-verified via `otool -D` - `actions/attest-build-provenance@v4.1.0` — keyless Sigstore provenance over every release artifact; `gh attestation verify` - `checksums.txt` covers both CLI and FFI assets; asset-count assertion bumped 3 → 8 - npm package stays CLI-only; Python/Swift/Go/Ruby/Node/C hosts pull dylib tarball directly from the Release - README gains a "Language bindings (FFI)" section with platform→artifact table **Docs** - `skills/agent-desktop-ffi/` — SKILL.md + build-and-link.md + ownership.md + threading.md + error-handling.md - `docs/solutions/best-practices/` — two new solution docs (deterministic build-artifact marker, identity fingerprint against OS reorder) **Verification** - `cargo clippy --all-targets -- -D warnings` — clean - `cargo test --lib --workspace` — 139 passed (5 suites) - `cargo test -p agent-desktop-ffi --tests` — 87 passed (4 suites, including new `c_header_compile` C-ABI harness) - macOS PR CI green on multiple runs; latest: 24561160455 - Python `ctypes.CDLL` round-trip from a freshly extracted FFI tarball confirmed; adapter lifecycle + `ad_list_apps` enumeration clean; `install_name` survives the tarball Closes #22 (superseded). |
||
|
Lahfir
|
c17f2fae7a
|
feat: progressive skeleton traversal with ref-rooted drill-down (#20)
* feat: add data model foundation for progressive skeleton traversal
Add children_count, root_ref, skeleton fields to core types. Add
get_subtree() to PlatformAdapter trait. Add remove_by_root_ref() and
write-side size check to RefMap. No behavioral changes yet.
* feat: implement progressive skeleton traversal with ref-rooted drill-down
Add --skeleton and --root flags to snapshot command for token-efficient
accessibility tree exploration. Skeleton mode clamps depth to 3 levels
and annotates truncated containers with children_count, allowing AI
agents to discover regions before drilling into them. Named containers
at skeleton boundaries (via name or description) receive refs as
drill-down targets. The --root flag starts traversal from a previous
ref with scoped invalidation — only refs from that drill-down are
replaced on re-drill.
Key changes:
- New ref_alloc.rs: shared ref helpers (INTERACTIVE_ROLES, actions_for_role,
ref_entry_from_node, is_collapsible) extracted from snapshot.rs
- New snapshot_ref.rs: drill-down logic with DrillDownConfig, scoped
invalidation via root_ref tagging on RefEntry
- macOS count_children() uses raw CFArrayGetCount without materializing
AXElement wrappers for performance at skeleton boundaries
- RefMap write-side size check prevents >1MB files
- Skeleton anchors consider both name and description for Electron compat
* fix: mention --skeleton in STALE_REF error suggestion
* docs: document --skeleton and --root flags in skill reference
* docs: update phases.md and CLAUDE.md for progressive skeleton traversal
Add skeleton traversal as Phase 1 objective P1-O10. Document --skeleton
and --root flags, get_subtree() trait method, new core modules, and
platform-agnostic notes for Phase 2/3. Update risk mitigations and
performance optimizations table.
* docs: make progressive skeleton traversal the default agent workflow
Update SKILL.md observe-act loop to skeleton-first approach. Add
progressive skeleton traversal as the primary workflow pattern. Update
anti-patterns, key principles, and command quick reference to lead
with --skeleton + --root. Full snapshot remains documented as fallback
for simple apps.
* fix: preserve skeleton drill-down anchors
* fix: preserve drill-down refs across skeleton re-snapshots
Skeleton snapshots now load the existing refmap and remove only
skeleton-level refs (root_ref: None), preserving drill-down refs
accumulated via --root. Previously, build() always created a fresh
RefMap, breaking the skeleton → drill → act → skeleton(verify) workflow
by wiping all drill-down refs on the verify step.
* fix: preserve drill-down depth for root snapshots
* fix: verify AX action effect on Electron elements before trusting success
Chromium's AX implementation returns kAXErrorSuccess for AXPress,
AXConfirm, etc. but only toggles ARIA state without firing DOM event
handlers. This caused the click chain to short-circuit on false
positives, preventing CGClick from ever being reached.
The fix detects web elements via AXWebArea ancestor walk, then verifies
each AX action actually had a DOM effect by comparing focused element
pointers before and after. When all AX methods produce no real effect,
CGClick fires as the genuine last resort.
Native elements are unaffected — the original verified_press behavior
is preserved in a separate code path.
* chore: ignore .context/ for local tooling artifacts
* style: collapse web_action_had_effect signature to single line
* test: cover RefMap save oversize rejection
Extract the serialize+size-check step into a private helper so the 1MB
write rejection path can be unit-tested without filesystem I/O. Also
moves the size check above the directory creation in save() so an
oversized refmap no longer creates ~/.agent-desktop on rejection.
* test: assert stale-ref suggestion mentions --skeleton
Locks in the Phase 4 polish requirement that STALE_REF errors guide
agents back to a skeleton refresh, not just a plain snapshot.
* test: cover skeleton-to-drill-down counter continuity
Asserts that skeleton refs (@e1..@e10) survive a scoped invalidation
of @e3, that drill-down refs allocated after the skeleton continue from
@e11 instead of resetting, and that remove_by_root_ref drops only the
drill-down children.
* test: cover --root + --surface rejection at execute() boundary
Adds a NoopAdapter that uses every PlatformAdapter trait default to
exercise execute()'s validation guards without standing up a real
adapter. Verifies that combining --root with a non-Window surface
returns INVALID_ARGS, and that the Window surface does not trigger the
guard.
* test: add filesystem-redirected integration tests for run_from_ref
Adds a thread-local HOME override + RAII HomeGuard so RefMap::save and
RefMap::load can be exercised against an isolated temp directory without
env-var racing across parallel tests. Also adds a StubAdapter that
implements PlatformAdapter with a canned subtree response so the full
run_from_ref drill-down flow can be unit-tested without standing up
macOS AX state.
Closes seven Phase 3 plan acceptance items in one pass:
- save+load roundtrip with HOME override
- oversize save rejection preserves previous file on disk
- run_from_ref returns subtree and persists drill refs
- stale root ref → STALE_REF with skeleton suggestion
- re-drill replaces drill refs only, counter continues
- multiple drill-downs from @e1 and @e2 coexist
- empty subtree drill-down produces no new refs
* test: add golden fixtures for skeleton output and drill-down refmap
Adds two committed JSON fixtures under tests/fixtures/ plus matching
unit tests that build the same input trees, run them through
allocate_refs / run_from_ref, and assert the produced ref ids,
parent-child layout, and root_ref tagging match the golden expectations.
Locks in the JSON shape so future serialization changes have to be
deliberate.
* fix: bound build_subtree recursion on Electron wrapper chains
Anonymous AXGroup/AXGenericElement wrappers do not advance the semantic
depth counter (web-wrapper depth-skip), so a long chain of nested
wrappers could recurse arbitrarily deep without ever hitting either
max_depth or ABSOLUTE_MAX_DEPTH. The previous code checked
`depth >= ABSOLUTE_MAX_DEPTH` against the semantic depth, which stayed
at 0 throughout a wrapper chain. On pathological Electron trees this
risked stack exhaustion before any cap engaged.
Add a separate `raw_depth` parameter that always increments and use it
for the absolute cap. Semantic `depth` still drives `max_depth` and the
skeleton boundary so the wrapper-flattening behavior is preserved on
normal trees.
Also drops the long-unused `_include_bounds` parameter; bounds
filtering happens in `allocate_refs` in core, not here.
* chore: add pre-commit hook running fmt + clippy + tests
Mirrors the CI quality gates (cargo fmt --check, cargo clippy
--all-targets -- -D warnings, cargo test --lib --workspace) so a
failing commit never reaches origin. Skips automatically when no
Rust/TOML files are staged. Bypass with --no-verify or SKIP_PRECOMMIT=1
when a non-Rust hotfix is genuinely needed.
Hook lives under .githooks/ and is opt-in per clone:
git config core.hooksPath .githooks
Setup instructions added to CLAUDE.md.
* fix: prevent orphaned drill-down refs leaking across skeleton refresh
remove_skeleton_refs() previously dropped every entry whose root_ref
was None and kept every entry whose root_ref was Some. After a series
of skeleton refreshes that pattern leaked drill-down refs whose root
anchor had already been removed: they could never be cleaned up by a
future remove_by_root_ref(target) because target referred to a
non-existent anchor, and they accumulated until the 1MB write guard
fired.
Restructure the function to keep only:
- skeleton anchors that have at least one drill-down pointing at them
- drill-down refs whose root anchor is among the kept anchors
Unreferenced anchors are still dropped (the original intent), and
orphaned drill-downs are dropped at the same time so the refmap can
no longer accumulate dead entries.
Updated existing test to assert the new keep-when-referenced semantics
and added a regression test for the orphan-drilldown leak path.
* fix: align resolve traversal with snapshot child-attribute set
find_element_recursive previously walked AXChildren first then fell
back to AXContents only, and resolve_element_name only derived its
fallback label from AXChildren. Snapshot building, however, uses the
full child_attributes() list (AXChildren, AXContents,
AXChildrenInNavigationOrder) via copy_children. That asymmetry meant
refs minted for containers exposed only through AXChildrenInNavigationOrder
appeared in the snapshot output but produced false STALE_REF errors on
drill-down or action commands because the resolver could not find them
again.
Route both call sites through child_attributes() so resolution sees
the same children as snapshotting.
* fix: bypass AXConfirm on web elements and add skeleton anchors to drill-down
* fix: compare AX elements via CFEqual in web_action_had_effect
AXUIElementCopyAttributeValue follows the CoreFoundation Create rule
and returns a freshly-allocated CF object on every call, so raw
pointer equality (`before.0 != after.0`) between two separate copies
of AXFocusedUIElement was ALWAYS true even when the focused element
was unchanged. That made Step 1 (AXPress) of activate_web_element
appear successful every time and left Steps 2-4 of the escalation
chain (AXConfirm bypass, child actions, CGClick fallback) dead code
on web elements.
Replace the pointer comparison with CFEqual, which compares
accessibility element identity rather than heap addresses.
* refactor: unify allocate_refs across snapshot and drill-down paths
allocate_refs in snapshot.rs and allocate_refs_with_root in
snapshot_ref.rs were near-exact copies that differed only in whether
each allocated ref carried a root_ref tag. The duplication meant any
bug fix or behavior change in one path risked silently drifting from
the other (bot flagged this after the first round of fixes added even
more duplicated skeleton-anchor logic to allocate_refs_with_root).
Move the shared logic into ref_alloc.rs behind a new RefAllocConfig
struct with Option<&str> for the root_ref_id, and delete the
snapshot_ref.rs copy. Both snapshot::build and snapshot_ref::run_from_ref
now route through the same function with their respective configs.
append_surface_refs and the existing unit tests in both modules are
updated to use the shared function + config.
* chore: drop with_root from drill test names after allocator unification
The snapshot_ref tests kept their original test_allocate_refs_with_root_*
names after the allocator dedupe even though allocate_refs_with_root no
longer exists. Rename them to test_drill_alloc_* so grep for
allocate_refs_with_root returns zero matches and nobody assumes a second
allocator path exists.
* docs: compound DRY ref-allocator dedupe into knowledge base
Adds docs/solutions/best-practices/deduplicate-ref-allocator-via-config-struct-2026-04-14.md
documenting the root cause, guidance, consequences, trigger conditions,
and before/after of the allocate_refs / allocate_refs_with_root
duplication that landed on PR #20 and was unified in
|
||
|
Lahfir
|
a19c1b5132 |
feat: add electron/web app compatibility for accessibility tree traversal
Skip depth budget for non-semantic AXGroup/AXGenericElement wrappers with empty name and value, allowing default --max-depth 10 to find 100+ refs in Electron apps like Slack and VS Code (previously found only 3). Raise resolver search depth from 20 to ABSOLUTE_MAX_DEPTH (50) so ref resolution succeeds for deeply nested Electron elements. Fix surface detection to check if focused window itself is the target surface (Electron reports dialogs as focused window, not as children) and check both AXRole and AXSubrole for surface matching. Update Phase 2 (Windows) and Phase 3 (Linux) docs with equivalent web/Electron compatibility patterns for their respective APIs. |
||
|
Lahfir
|
8aa672c668 |
refactor: resolve code review findings from notification PR
- fix(core): wait --notification uses index-diff detection instead of count, passes text filter, checks deadline before sleep - fix(core): dismiss-all-notifications uses single NC session with batch adapter method and reports individual failures - refactor(macos): extract dismiss_entry helper to eliminate duplicate dismiss logic between single and batch operations - refactor(macos): restrict nc_session visibility to pub(crate), use absolute paths for pgrep/osascript, add bounded wait with kill fallback - refactor(macos): single-pass is_notification_group check using matches! macro - refactor: extract notification CLI args and dispatch into separate files to keep cli_args.rs and dispatch.rs under 400 LOC - refactor: rename DismissAllNotificationsArgs to DismissAllNotificationsCliArgs for consistency - fix(core): remove unused timestamp field from NotificationInfo - fix: remove poll_interval_ms from wait command (hardcode 500ms) - docs: update phases.md with completed notification status and cross-platform reference notes |
||
|
Lahfir
|
844e964b82 | docs: update phases with notification and tray trait methods | ||
|
Lahfir
|
03268ce18e | docs: add comprehensive phase roadmap | ||
|
Lahfir
|
50335f4576 | chore: remove internal docs from tracking and fix gitignore | ||
|
Lahfir
|
a73fe2e6c1 | docs: add architecture diagram to README | ||
|
Lahfir
|
ad91cd32cf |
feat: add Claude Code skills for agent-desktop automation
Graph of skills covering all 50 commands with platform-specific sub-skills: - Main index skill with core concepts, ref system, JSON contract - Observation, interaction, and system command references - Common workflow patterns (forms, menus, dialogs, drag-drop) - macOS platform skill (permissions, AX API, troubleshooting) - /desktop slash command for quick reference - PRD addendum for skill maintenance across phases |
||
|
Lahfir
|
b2c34f6ec5 |
Revert "docs: add platform crate folder structure convention and track plans"
This reverts commit
|
||
|
Lahfir
|
775892ee8e |
docs: add platform crate folder structure convention and track plans
Add Platform Crate Folder Structure section to CLAUDE.md enforcing consistent subfolder layout (tree/, actions/, input/, system/) across all platform crates. Update smart AX-first click chain plan with new file paths reflecting the subfolder structure. Un-ignore docs/plans/ and docs/brainstorms/ so design documents are tracked. |
||
|
Lahfir
|
3a796023f0 | style: apply cargo fmt to all files | ||
|
Lahfir
|
29d6cbaac4 | docs: check off all Phase 1 Must-Fix agent-native items | ||
|
Lahfir
|
a346f242c2 |
feat: Phase 1 foundation — workspace scaffold, core engine, macOS adapter, 31 commands
Implements the complete agent-desktop Phase 1 specification:
- Workspace: 5-crate layout (core, macos, windows/linux stubs, binary)
- Core: AccessibilityNode, Action, ErrorCode, PlatformAdapter trait, RefMap
with atomic writes, SnapshotEngine with depth-first ref allocation
- macOS adapter: AXUIElement tree traversal, action execution, input
synthesis via CGEvent, screenshot via CGWindowListCreateImage, clipboard
via pbpaste/pbcopy, window listing and app management via osascript
- 31 CLI subcommands via clap derive: snapshot, find, screenshot, get, is,
click, double-click, right-click, type, set-value, focus, select, toggle,
expand, collapse, scroll, press, launch, close-app, list-windows,
list-apps, focus-window, clipboard-get/set, wait, status, permissions,
version, batch
- Windows/Linux: not-supported stubs ready for Phase 2 implementation
- JSON output contract: {version,ok,command,data} envelope with structured
error payloads including SCREAMING_SNAKE_CASE error codes
- Ref system: @e{N} sequential refs for interactive elements, stored at
~/.agent-desktop/last_refmap.json with 0o600/0o700 permissions
- CI: GitHub Actions macOS runner with dependency isolation check, clippy,
unit tests, release build, and 15MB binary size gate
|