mirror of
https://github.com/lahfir/agent-desktop.git
synced 2026-08-09 08:27:25 +00:00
4f7bad337c
* docs: add sub-phase 2.8 input synthesis plan * docs: harden the 2.8 input synthesis plan after adversarial review * docs: give every 2.8 deferral a receiving owner in phases.md * feat: measure windows input synthesis gaps (area 20) Probe the unmeasured SendInput and token facts 2.8 depends on so coordinate, abort-drag, double-click, integrity, and cost branches are settled before the adapter InputOps land. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows mouse_event via SendInput Give bare-coordinate hover/click/wheel a real physical path with primary-vs-virtual-desktop normalization, FFI held-edge guards, and a release-safe modifier hold around multi-click. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows drag with origin release guard Port the macOS abort-state contract onto SendInput so an interrupted drag posts corrective release at the origin and reports delivered_unverified instead of claiming a clean failure. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: implement windows keyboard synthesis and blocked combos Ship chord and UTF-16 type_text primitives with release guards, macOS key-name parity, and a Windows-specific dangerous-combo list so press/type can land without stuck keys or silent session-kill shortcuts. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: detect uipi elevation before physical input Compare caller and target integrity levels so a Medium-into-High SendInput attempt fails closed as PERM_DENIED instead of reporting a silent no-op as success. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: wire windows physical execute_action legs Replace the 2.7 not-supported stubs for type, press, multi-click, and right-click with SendInput synthesis gated by focus, elevation, and headed policy. Co-authored-by: Cursor <cursoragent@cursor.com> * test: pin windows physical input envelope parity Lock AdapterError and ActionResult wire shapes for the physical legs against the macOS disposition contract, and cite the A20-6 hot-path cost baseline already captured in area 20. Co-authored-by: Cursor <cursoragent@cursor.com> * feat: dogfood windows input synthesis and harden delivery Prove the physical path on Notepad and ScratchForms under the corpus safety envelope, and fix the dogfood-found root-window foreground gate plus SendInput absolute normalization and drag pickup timing. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: align windows input synthesis docs with shipped evidence Restate phases, concepts, skill wording, and ledger rows so 2.8 reads as what shipped, including UIPI detection closure and the type command divergence handed to 2.15. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: harden windows input abort guards after review Strip plan decision ids from crates so CI phase-reference scan passes, arm a click release guard around each down/up, and re-verify keyboard focus at every type_text chunk boundary. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor: give the mouse input constructors and release-guard state one home button_input existed verbatim in four files and move_input in three, so the corrective batch an abort depends on could drift from the normal post. The click and drag guards also carried the same armed-and-counted state machine and the same delivery report, differing only in their recovery hint. Both now have a single home: mouse_send.rs owns the SendInput surface - the MOUSEEVENTF_* flags with their compile-time assertions against windows-sys, plus the two input constructors - and release_state.rs owns the guard state and the delivery report it produces. Each guard keeps its own Drop, because only the guard knows whether the corrective post is a button-up in place or a move back to the origin first. * fix: resolve the host window before refusing a physical click target_window_is_foreground read the leaf element's NativeWindowHandle, which UIA reports as 0 - success, not failure - for every element that is not itself a window. That is the normal shape for WPF, WinUI, UWP and Chromium content, so double-click, triple-click and right-click were refused before injection on most modern desktop UI. The handle is now resolved the way the rest of the crate resolves it, by climbing to the first ancestor that owns one. Also from the same review pass: - mouse-wheel dropped the caller's modifiers, so the documented --modifiers shift horizontal-scroll chord silently degraded to a plain vertical scroll while still reporting success. - The drag preflight reserved one pickup delay where the sequence spends two, admitting drags that then aborted mid-gesture with the button down. Both delays now come from one constant. - The skill docs carried plan decision ids and sub-phase numbers, and they are include_str!d into the binary, so agent-desktop skills served them to agents. The phase-reference scan now covers skills/ for exactly that reason and caught a second instance in SKILL.md. * fix: close the remaining input-synthesis review findings - The click sequence counted delivered events per click and let the inter-click gap fail through a bare ?, so a --count 3 that aborted late reported two events, or none at all with an unknown disposition. Accounting now spans the sequence while cleanup stays per click. Wheel chunks report the same way: a partial scroll says how much landed. - mouse_modifier opened a second SendInput keyboard path with its own fake sink, so the one-seam invariant keyboard_send documents was already false. It now posts through that seam and reuses its VK map. - type_text's focus gate and per-chunk re-verify were exercised through a cfg(test) clone that omitted the policy gate, the preflight and the real verifier, so deleting either guard left the suite green. The gate is one production function now, with the verifier injected, and a test drives it with a verifier that fails mid-sequence. - Two drag tests asserted a std-library identity and re-typed the production step arithmetic. They drive synthesize_drag and read the posted event stream instead. Inverting the first showed the zero-dwell early return was dead - the loop guard already covered it - so it is gone. - elevation's module doc blamed unavailable token manufacture for the unproven cross-boundary effect; the captures record manufacture working with no higher-integrity target to aim at. keyboard's docs claimed its primitives were unreachable from execute_action, which calls both. * docs: capture the input-synthesis learnings and correct what the run overstated Two new solutions docs and one extended: - An enforcement gate must cover everything the binary embeds. The phase-reference scan read crates/ and src/ for .rs files while the skill markdown it never looked at was include_str!d into the binary, so plan ids were being served to agents by the skills command. Widening the scope immediately found a ninth instance nobody had reported. - A zero success value is not the answer you asked for. UIA returns NativeWindowHandle 0 as a success for any element that is not a window, and the crate already had the climb that treats it as keep-looking; the physical-click gate re-derived a naive version and refused delivery on WPF, WinUI and Chromium. - The cannot-fail taxonomy gains the cfg(test) re-implementation shape, and a prevention note to invert one production site per run: two mutations at once made a test fail on the other one's arithmetic and hid the failure that mattered. The dogfood report claimed the multi-click path more broadly than it measured it. Both multi-click judgements ran against HWND-bearing targets, which is precisely where the defect was invisible, so J4 now states its scope and the residual records that a non-HWND judgement is still owed. * docs: judge multi-click on a non-HWND target and close the residual The original run judged multi-click only against WinForms and Notepad, both of which own a window handle - the one shape where the foreground gate's leaf-handle read could not fail. J10 runs the same WPF button with the pre-fix and fixed binaries back to back: the first reports ACTION_FAILED and not_delivered with the app's click counter untouched, the second lands and the counter advances to action:2, read back independently of the envelope. Two correct refusals fell out of the same run and are recorded with it: the occlusion gate naming the terminal window that actually covered the fixture, and the private-file guard rejecting a store an elevated run had left owned by the Administrators group rather than the user. * fix: block modifier supersets of a dangerous shortcut Exact canonical-string matching let alt+shift+tab through to SendInput. That is the reverse task switcher and takes the foreground exactly as the alt+tab it extends, which is the harm the list exists to prevent. A test asserted the old behaviour as correct, reasoning that an extra modifier makes a shortcut unrelated; shift does not neutralize alt+f4 and does not neutralize alt+tab, so the premise was wrong and the test is gone. Matching is now key equality plus modifier superset. It over-blocks a few combinations nobody presses, and that trade is deliberate: a wrongly blocked combo costs one --force, a wrongly allowed one moves input to another window mid-run. macOS answers the same problem by enumerating variants (it lists cmd+q and cmd+shift+q separately), so the two adapters now decide danger by different rules. Recorded as a settlement item for the hardening gate rather than changed here, since the macOS crate is the GA line. --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|---|---|---|
| .. | ||
| brainstorms | ||
| dogfood-reports | ||
| plans | ||
| solutions | ||
| architecture.excalidraw | ||
| architecture.html | ||
| architecture.png | ||
| example.html | ||
| example.png | ||
| faq.md | ||
| json-output.md | ||
| phases.md | ||
