Commit graph

75 commits

Author SHA1 Message Date
Lahfir
3881bc82bd fix(macos): restore frontmost app after notification center interaction
NcSession now captures the frontmost application before opening
Notification Center and reactivates it on close, preventing the
source app (e.g. Script Editor) from stealing focus during dismiss.
2026-02-27 14:30:31 -08:00
Lahfir
27ef4f34c0 fix(macos): remove AXPress from dismiss action list
AXPress "clicks" the notification body (opening the source app) but
does not actually dismiss it. Removing it from the initial action list
forces the code to fall through to the hover-and-close-button strategy,
which reliably removes the notification from Notification Center.
2026-02-27 14:25:15 -08:00
Lahfir
8aa672c668 refactor: resolve code review findings from notification PR
- fix(core): wait --notification uses index-diff detection instead of
  count, passes text filter, checks deadline before sleep
- fix(core): dismiss-all-notifications uses single NC session with
  batch adapter method and reports individual failures
- refactor(macos): extract dismiss_entry helper to eliminate duplicate
  dismiss logic between single and batch operations
- refactor(macos): restrict nc_session visibility to pub(crate), use
  absolute paths for pgrep/osascript, add bounded wait with kill
  fallback
- refactor(macos): single-pass is_notification_group check using
  matches! macro
- refactor: extract notification CLI args and dispatch into separate
  files to keep cli_args.rs and dispatch.rs under 400 LOC
- refactor: rename DismissAllNotificationsArgs to
  DismissAllNotificationsCliArgs for consistency
- fix(core): remove unused timestamp field from NotificationInfo
- fix: remove poll_interval_ms from wait command (hardcode 500ms)
- docs: update phases.md with completed notification status and
  cross-platform reference notes
2026-02-27 12:42:46 -08:00
Lahfir
844e964b82 docs: update phases with notification and tray trait methods 2026-02-27 11:57:18 -08:00
Lahfir
de651737f1 refactor(macos): headless-first dismiss with cursor as last resort
Try AXDismiss/AXRemoveFromParent/AXPress on the notification group
element before falling back to cursor hover. Extract try_dismiss_button
helper. Cursor-based hover only triggers if all headless strategies
fail.
2026-02-27 11:37:47 -08:00
Lahfir
979758538f fix(macos): use pgrep and async osascript for NC lifecycle
CGWindowList requires Screen Recording permission on macOS Sequoia to
read other processes' window names, and SystemUIServer blocks AX access
to its menu bar (crashes with Obj-C foreign exception). Switch to pgrep
for PID lookup and osascript clicking ControlCenter's Clock menu bar
item, spawned async to avoid blocking.
2026-02-27 11:33:36 -08:00
Lahfir
2374330bb1 test: add notification error and filter matching tests
Add error code serialization and notification_not_found tests in core.
Add matches_filters unit tests in macOS list module covering app,
text, combined, and no-filter cases.
2026-02-27 10:53:15 -08:00
Lahfir
53d697d522 feat(macos): implement dismiss and notification action commands
Dismiss synthesizes mouse hover to reveal close button (Sequoia), then
finds and AXPress it. Falls back to AXPress on notification group.
Notification action finds matching AXButton by name and presses it.
Both reuse shared list_entries for AX element access.
2026-02-27 10:51:13 -08:00
Lahfir
53549a384e feat(macos): implement notification list via AX tree traversal
Traverses NC's AX tree to extract notification info (app, title, body,
actions). Handles variable nesting depth across Sonoma/Sequoia via
recursive heuristic matching. Supports --app, --text, and --limit
filters applied during traversal for early termination.
2026-02-27 10:47:36 -08:00
Lahfir
0d55c21de0 feat(macos): add NC session RAII guard and notification adapter wiring
Implements NcSession for safe Notification Center lifecycle management
(open via AX press on clock, close via Escape, with Drop fallback).
Wires list_notifications, dismiss_notification, and notification_action
adapter methods to placeholder stubs pending Phase 3/4 implementation.
2026-02-27 10:45:25 -08:00
Lahfir
c5b05bab60 feat: add notification command types, adapter trait, and CLI wiring
Core types (NotificationInfo, NotificationFilter), 3 adapter methods
with not_supported defaults, NotificationNotFound error code, 4 command
handlers, CLI args with index>=1 validation, dispatch arms, wait
--notification extension, and batch_dispatch split to stay under 400 LOC.
2026-02-27 10:40:07 -08:00
Lahfir
03268ce18e docs: add comprehensive phase roadmap 2026-02-25 01:50:57 -08:00
Lahfir
50335f4576 chore: remove internal docs from tracking and fix gitignore 2026-02-24 06:24:26 -08:00
Lahfir
a73fe2e6c1 docs: add architecture diagram to README 2026-02-24 06:22:52 -08:00
Lahfir
74f2c4f977 docs: rewrite README for clarity and add npm installation 2026-02-24 06:01:57 -08:00
Lahfir
38633518e3
Merge pull request #10 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release 0.1.6
2026-02-24 05:56:49 -08:00
github-actions[bot]
5a0adb7fd6
chore(main): release 0.1.6 2026-02-24 13:56:03 +00:00
Lahfir
d4197e8f6f fix: handle null bounds in refmap and improve sidebar click resolution
Rect deserialization now tolerates null/missing fields (defaults to 0.0),
preventing corrupt refmaps from breaking all subsequent click commands.
read_bounds rejects NaN/Inf at the source. Sidebar cells now resolve via
parent row AXSelected instead of falling through to CGClick. Extracted
chain step functions into chain_steps.rs to stay within 400 LOC limit.
2026-02-24 05:55:44 -08:00
Lahfir
a33b86649c
Merge pull request #9 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release 0.1.5
2026-02-23 04:43:39 -08:00
github-actions[bot]
9a2657556d
chore(main): release 0.1.5 2026-02-23 12:43:05 +00:00
Lahfir
7f3232a012
Merge pull request #8 from lahfir/refactor/centralized-ax-chain-executor
refactor: centralize AX chain executor with verbose logging
2026-02-23 04:42:47 -08:00
Lahfir
11f8da06e8 feat: add fallback chains for set-value, clear, focus, scroll-to, type and post-action state hints
Stage B of the centralized chain executor plan:
- set-value chain: direct AXSetValue → focus+AXSetValue
- clear chain: AXSetValue("") → focus+AXSetValue("") → Cmd+A+Delete
- focus chain: AXFocused → AXRaise → AXPress → AXSelected → CGClick
- scroll-to chain: AXScrollToVisible → walk-parents-scroll
- type: app-focus before typing, non-ASCII via clipboard paste (Cmd+V)
- post-action state hints: read element role/value/states after stateful
  actions (click, toggle, check, set-value, clear, type, expand, collapse)
- FocusThenSetDynamic chain step variant for focus-before-set patterns
- export copy_value_typed for reading numeric AXValues (checkboxes)
2026-02-23 04:41:12 -08:00
Lahfir
2a52d62106 fix: add dwell time before drag release for drop target recognition 2026-02-23 04:24:35 -08:00
Lahfir
e154cc0cc0 refactor: address code review findings
- dispatch.rs: 460 → 392 LOC by removing duplicate command_name(),
  using cmd.name() from cli.rs, and moving cli_surface_to_core to
  Surface::to_core() in cli_args.rs
- ElementCaps: trim from 8 to 3 used fields, remove has_action() and
  unused discovery calls (actions, role, has_children, pid, settable_value)
- chain.rs: deduplicate ChainStep/ChainDef/ChainContext out of cfg gates
- ax_helpers.rs: delete dead set_ax_string
2026-02-23 04:10:50 -08:00
Lahfir
c7316e8b51 feat: add structured verbose logging across all layers
Adds tracing::debug! calls throughout command and adapter layers,
activated via -v flag. Logs command dispatch, ref resolution, chain
step execution, clipboard/keyboard/mouse synthesis, and system ops.

Command layer logging (free for all platforms):
- dispatch: command name
- resolve: ref lookup with pid/role/name, match result
- tree: snapshot app/window/ref_count

Adapter layer logging (macOS, other platforms add their own):
- chain: step-by-step [N/total] with success/skip for each
- action: perform entry
- clipboard/keyboard/mouse: operation details
- system: app focus/launch/close, window ops, screenshots
2026-02-23 03:52:57 -08:00
Lahfir
4fe91e3a96 refactor: centralize AX chain executor with error suggestions and resilience
- Add ax_helpers.rs: 16 shared AX utility functions, eliminating ~150 LOC duplication
- Add chain.rs: declarative ChainStep executor with 9 step variants and 10s deadline
- Add chain_defs.rs: static chain definitions for click (14-step), right-click, expand, collapse
- Add discovery.rs: one-time ElementCaps query shared across chain steps
- Replace NSPasteboard subprocess calls with direct objc_msgSend FFI (zero new deps)
- Add .with_suggestion() to 16 high-priority error paths across 8 files
- Fix FocusThenAction to use retried variant for kAXErrorCannotComplete resilience
- Add relaxed stale ref matching (name-only fallback when bounds change)
- Delete activate.rs, absorb logic into chain_defs and ax_helpers (-388 net LOC)
2026-02-23 03:32:17 -08:00
Lahfir
30b9a4fc92
Merge pull request #7 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release 0.1.4
2026-02-23 02:40:57 -08:00
Lahfir
ada8f4f0ee chore: remove completed todo files and build artifacts 2026-02-23 02:04:54 -08:00
github-actions[bot]
f61194dd46
chore(main): release 0.1.4 2026-02-23 10:02:16 +00:00
Lahfir
ef45135087 feat: add agent-desktop skill for universal AI agent support
Installable via: npx skills add lahfir/agent-desktop
2026-02-23 02:02:00 -08:00
Lahfir
396e3e5abf revert: remove .claude directory from git tracking
Keep .claude/ fully ignored — skills are distributed via the
.skill package on GitHub Releases, not checked into the repo.
2026-02-23 01:52:25 -08:00
Lahfir
ad91cd32cf feat: add Claude Code skills for agent-desktop automation
Graph of skills covering all 50 commands with platform-specific sub-skills:
- Main index skill with core concepts, ref system, JSON contract
- Observation, interaction, and system command references
- Common workflow patterns (forms, menus, dialogs, drag-drop)
- macOS platform skill (permissions, AX API, troubleshooting)
- /desktop slash command for quick reference
- PRD addendum for skill maintenance across phases
2026-02-23 01:51:31 -08:00
Lahfir
572a2a3f25
Merge pull request #6 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release 0.1.3
2026-02-23 01:25:59 -08:00
github-actions[bot]
9932e352b7
chore(main): release 0.1.3 2026-02-23 09:25:38 +00:00
Lahfir
ebafb71603 perf: use curl for binary download in postinstall
Replace Node.js https.get with curl for 15x faster downloads.
GitHub's redirect chain causes excessive TLS handshake overhead
in Node's http client.
2026-02-23 01:25:22 -08:00
Lahfir
8f66a9346e fix: correct GitHub Release download URL and simplify tag format
Use v{version} tags instead of agent-desktop-v{version} by setting
include-component-in-tag: false in release-please config.
2026-02-23 01:20:16 -08:00
Lahfir
084fc8c960 fix: include README and CHANGELOG in npm package 2026-02-23 01:16:15 -08:00
Lahfir
8cbca19504
Merge pull request #5 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release agent-desktop 0.1.2
2026-02-23 01:04:16 -08:00
github-actions[bot]
e58c0732c8
chore(main): release agent-desktop 0.1.2 2026-02-23 09:03:37 +00:00
Lahfir
91c76777cb fix: use macos-latest for both build targets
macos-13 Intel runners are no longer available. Cross-compile x86_64
on ARM runners instead, which Rust handles natively via --target.
2026-02-23 01:03:24 -08:00
Lahfir
dd5532c454
Merge pull request #4 from lahfir/release-please--branches--main--components--agent-desktop
chore(main): release agent-desktop 0.1.1
2026-02-23 01:00:06 -08:00
github-actions[bot]
dc9f2afddf
chore(main): release agent-desktop 0.1.1 2026-02-23 08:59:15 +00:00
Lahfir
0ab78dde0e fix: use simple release strategy for workspace version bumps
cargo-workspace plugin cannot parse version.workspace = true in member
crates. Switch to simple strategy with generic file updater using
x-release-please-version annotation in root Cargo.toml.
2026-02-23 00:57:28 -08:00
Lahfir
18fc50cca5
feat: add release automation with GitHub Releases and npm distribution
Automated release pipeline using release-please for gated Release PRs,
conventional commits for SemVer version bumps, and npm distribution
with postinstall binary download from GitHub Releases.
2026-02-23 00:55:45 -08:00
Lahfir
aa3b210708 style: fix cargo fmt check in activate.rs 2026-02-21 23:37:02 -08:00
Lahfir
48f8470948 feat: add structural hints to splitter columns in snapshots
Multi-column apps like Notes, Mail, and Finder bury the editor
textfield deep in the tree (e.g. ref 256 of 272). Agents picking
"the first textfield" grab the wrong one. Column hints let agents
distinguish sidebar vs content-list vs editor panes.
2026-02-21 17:28:49 -08:00
Lahfir
a349bd9bc0 docs: rewrite README with AX-first interaction chains and inline menu capture 2026-02-21 15:41:28 -08:00
Lahfir
cddc5d3547 feat: AX-first right-click chain with inline context menu capture
- Add 7-step right-click chain: AXShowMenu direct, focus-app +
  AXShowMenu, select + AXShowMenu, focus-element + AXShowMenu,
  parent AXShowMenu, child AXShowMenu, CGEvent last resort
- Key fix: AXShowMenu returns -25204 (CannotComplete) when app
  isn't frontmost; ensure_app_focused via AX resolves this
- Right-click command now returns full context menu tree inline
  with ref_ids on all menuitems for immediate agent interaction
- Remove surface-based menu detection (menu lives in regular tree)
- Clean up debug tracing from resolve.rs
2026-02-21 15:40:03 -08:00
Lahfir
595ccb6cc4 feat: 10-step scroll chain, focus guards, enhanced click chain, bounds fix
- Rewrite ax_scroll with 10-step AX-first chain (scroll-to-visible,
  increment/decrement, page scroll, value shift, sub-element press,
  focus child, select rows, keyboard, arrow keys, CGEvent last resort)
- Add ensure_app_focused() guard before all CGEvent calls
- Enhance smart_activate with AXScrollToVisible pre-step,
  AXShowAlternateUI, AXCustomActions, and keyboard space fallback
- Fix bounds_hash always null in RefMap (always read bounds in builder)
- Add right-click menu capture in response data
- Add append_surface_refs for post-action menu detection
2026-02-21 15:28:28 -08:00
Lahfir
2f495ffb69 fix: address code review findings (double-free, CF leaks, injection)
P1: fix double-free in find_scroll_area (el.clone() instead of AXElement(el.0))
P1: add input validation in close_app_impl to prevent AppleScript injection
P2: fix CFRelease leaks in focus_window_impl and press_for_app_impl
P2: remove duplicated copy_element_attr/copy_bool_attr from surfaces.rs
P3: replace is_attr_settable_pub wrapper with direct pub export
P3: extract TOGGLEABLE_ROLES constant, collapse try_parent_activation loop
P3: fix fragile super::super::builder path, tighten list_windows_impl visibility
2026-02-21 14:01:51 -08:00