Enables agents to detect UI changes between snapshots without
re-parsing the entire tree. Saves each snapshot to disk and diffs
against the previous one, reporting added/removed/modified nodes.
https://claude.ai/code/session_01WCp4PmZcXWCo7XeCfHPm3K
Documents the root cause (sequential AX IPC calls per node), the fix
(AXUIElementCopyMultipleAttributeValues batch fetching), measured wall-time
benchmarks across TextEdit/Finder/System Settings/Xcode, and the secondary
optimisations: FxHashSet cycle detection, streaming JSON serialisation, and
per-element messaging timeout.
AXShowMenu is the accessibility-native way to trigger a context menu on
an element without moving the cursor or posting CGEvents. After the
action, the resulting AXMenu appears as a child of the triggered element.
- actions.rs: RightClick tries AXShowMenu first, falls back to CGEvent
- surfaces.rs: context_menu_from_app checks AXFocusedUIElement's children
for AXMenu, then falls back to direct app children (Electron apps)
- list_surfaces_for_pid: surfaces context menus as 'context_menu' type
(distinct from menu-bar 'menu' entries)
AXMenus on AXApplication returns kAXErrorAttributeUnsupported (-25205)
for standard Cocoa apps. Menu bar menus are always children of their
AXMenuBarItem; the only reliable open-state indicator is AXSelected=true
on that item.
- open_menubar_menu: scan AXMenuBar.AXChildren for AXMenuBarItem with
AXSelected=true, return its AXMenu child
- context_menu_from_app: kept as fallback for Electron-style apps that
do expose context menus as AXMenu in AXApplication.AXChildren
- list_surfaces_for_pid: reports menu title and item_count when open
- is_menu_open: checks both paths
Add snapshot --surface flag (menu/sheet/popover/alert/focused/window)
for direct O(1) AX attribute reads rather than full-tree traversal.
Add wait --menu/--menu-closed for polling-based context-menu gate.
Add list-surfaces command to enumerate open transient surfaces.
Remove all inline // comments from macos crate per 400-LOC/no-comment rule.
- tree: window_element_for() starts traversal from correct AXWindow
element (matching by title) instead of app root, fixing mixed-window
trees and 'disabled group' noise in Electron apps
- tree: AXUIElementCopyMultipleAttributeValues batch fetch (2.5x faster)
- tree: copy_ax_array() CFRetains each element before CFArray drops,
fixing the dangling-pointer bug that returned kAXErrorInvalidUIElement
- roles: add AXApplication->application and AXSplitGroup->splitter
- actions: CGEvent is now primary click mechanism (AXPress is best-effort
first try); fixes clicks in Electron/web apps that don't support kAXPressAction
- actions: DoubleClick uses MOUSE_EVENT_CLICK_STATE=2, RightClick uses
RightMouseDown/Up, Scroll sets event.set_location(element_center)
- screenshot: rewrite using screencapture CLI + CGWindowListCopyWindowInfo
to find the largest CGWindowID for the app PID; resolves app/window_id
to pid before calling adapter (was producing all-zero data placeholder)
- screenshot: ScreenshotTarget::Window now carries pid (i32) instead of
our hash string; commands/screenshot.rs resolves via list_windows
- input: add 8ms inter-keystroke delay in synthesize_text so apps can
process events (was truncating long strings)
- snapshot: return WindowInfo in SnapshotResult; include app and window
fields in JSON output per contract
- find: switch to snapshot::run so found refs are persisted to refmap
- dispatch: implement dispatch_batch_command covering all 29 commands
(was stub returning 'not yet implemented' for every batch sub-command)
