fix: report a launch from what the process did, not from what is missing

Four defects in the launch path, all the same shape as the one this branch
set out to fix: an answer given without observing the thing it describes.

A process that exits during the launch left no window-server record, and
that absence was read as "finished starting up", so a launch that killed
its own target answered ok:true with no window. The window server keeps
no record of a process that exited and none of one that has not
registered yet, so libproc now decides which happened: a process that is
gone ends the launch with APP_UNRESPONSIVE.

--activate asks the application to present a window, so the wait now runs
to the caller's timeout instead of stopping at the startup grace. The
grace still bounds the path that asks for nothing, which is what kept a
plain launch from waiting on an event it never caused.

Minimized windows are the user's windows. Dropping them alongside the
bookkeeping panels reported an application as having no window while its
window sat in the Dock; only panels that were never meant to be seen are
dropped now.

The app field reported the requested identifier when launching and the
display name when attaching, so the same field meant two things. A launch
by bundle id answered com.apple.TextEdit where an attach answered TextEdit.

BREAKING CHANGE: the response envelope is version 2.3. The launch payload
changed shape in this release and a windowless launch stopped being an
error, which the envelope has to announce.
This commit is contained in:
Lahfir 2026-08-10 01:38:43 -07:00
parent c015bf1efb
commit dd853957c4
12 changed files with 125 additions and 30 deletions

View file

@ -286,7 +286,7 @@ Every command produces a response envelope:
```json
{
"version": "2.2",
"version": "2.3",
"ok": true,
"command": "snapshot",
"data": {
@ -302,7 +302,7 @@ Error responses:
```json
{
"version": "2.2",
"version": "2.3",
"ok": false,
"command": "click",
"error": {

View file

@ -4,7 +4,7 @@ use serde_json::Value;
use crate::recovery_hint::RecoveryHint;
use crate::{AppError, DeliverySemantics, ErrorCode, RetryDisposition};
pub const ENVELOPE_VERSION: &str = "2.2";
pub const ENVELOPE_VERSION: &str = "2.3";
/// Structured output envelope used by the CLI and future programmatic transports.
#[derive(Debug, Serialize)]

View file

@ -1421,7 +1421,7 @@ AdResult ad_execute_by_ref_timeout(const struct AdAdapter *adapter,
* to disk, and writes the JSON envelope into `*out`.
*
* The JSON shape matches `agent-desktop snapshot`:
* `{"version":"2.2","ok":true,"command":"snapshot","data":{"app":"...","window":{...},"ref_count":N,"snapshot_id":"...","tree":{...}}}`.
* `{"version":"2.3","ok":true,"command":"snapshot","data":{"app":"...","window":{...},"ref_count":N,"snapshot_id":"...","tree":{...}}}`.
*
* **`*out` ownership and error behaviour:**
* - On success (`AD_RESULT_OK`): `*out` is a heap-allocated JSON string with `"ok":true`.

View file

@ -15,7 +15,7 @@ use std::ptr;
/// to disk, and writes the JSON envelope into `*out`.
///
/// The JSON shape matches `agent-desktop snapshot`:
/// `{"version":"2.2","ok":true,"command":"snapshot","data":{"app":"...","window":{...},"ref_count":N,"snapshot_id":"...","tree":{...}}}`.
/// `{"version":"2.3","ok":true,"command":"snapshot","data":{"app":"...","window":{...},"ref_count":N,"snapshot_id":"...","tree":{...}}}`.
///
/// **`*out` ownership and error behaviour:**
/// - On success (`AD_RESULT_OK`): `*out` is a heap-allocated JSON string with `"ok":true`.

View file

@ -107,21 +107,31 @@ fn bridge_error(operation: &str, status: u8, delivery_started: bool) -> AdapterE
.with_disposition(disposition)
}
/// Where a process is in its startup. `NoRecord` covers both a process that
/// exited and one that has not registered with the window server yet, so it
/// answers neither question on its own and the caller has to ask libproc which
/// of the two it is.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum StartupState {
Starting,
Finished,
NoRecord,
}
/// An application that finished starting up has already created whatever
/// windows its launch produces. `None` means the answer is unavailable, which
/// is not the same as "no window is coming".
/// windows its launch produces.
#[cfg(target_os = "macos")]
pub(crate) fn finished_launching(pid: i32) -> Option<bool> {
pub(crate) fn startup_state(pid: i32) -> StartupState {
match unsafe { agent_desktop_app_finished_launching(pid) } {
0 => Some(false),
1 => Some(true),
_ => None,
0 => StartupState::Starting,
1 => StartupState::Finished,
_ => StartupState::NoRecord,
}
}
#[cfg(not(target_os = "macos"))]
pub(crate) fn finished_launching(_pid: i32) -> Option<bool> {
None
pub(crate) fn startup_state(_pid: i32) -> StartupState {
StartupState::NoRecord
}
#[cfg(target_os = "macos")]

View file

@ -65,10 +65,12 @@ fn settled_window(
if let Some(window) = exact_window(pid, process_instance, deadline)? {
return Ok(Some(window));
}
if options.timeout_ms == 0 || grace_over(grace_ends_at, Instant::now()) {
if options.timeout_ms == 0
|| (!options.activate && grace_over(grace_ends_at, Instant::now()))
{
return Ok(None);
}
if grace_ends_at.is_none() && startup_finished(pid) {
if grace_ends_at.is_none() && startup_finished(pid, process_instance)? {
grace_ends_at = Instant::now().checked_add(STARTUP_GRACE);
}
let remaining = deadline.remaining();
@ -80,11 +82,35 @@ fn settled_window(
}
}
/// An unreadable startup state ends the wait rather than extending it, because
/// a process that cannot answer is not one whose windows are worth waiting for.
/// Ends the wait when the application has created whatever windows its launch
/// produces. The window server keeps no record of a process that exited, and
/// none of one that has not registered yet, so libproc decides which happened:
/// a process that is gone ends the launch with an error rather than an answer
/// about windows it will never open.
#[cfg(target_os = "macos")]
fn startup_finished(pid: i32) -> bool {
crate::system::appkit_bridge::finished_launching(pid).unwrap_or(true)
fn startup_finished(pid: i32, process_instance: &str) -> Result<bool, AdapterError> {
use crate::system::appkit_bridge::StartupState;
match crate::system::appkit_bridge::startup_state(pid) {
StartupState::Starting => Ok(false),
StartupState::Finished => Ok(true),
StartupState::NoRecord => {
if crate::system::process_identity::matches_instance(pid, process_instance)? {
Ok(true)
} else {
Err(launch_target_gone(pid))
}
}
}
}
#[cfg(target_os = "macos")]
fn launch_target_gone(pid: i32) -> AdapterError {
AdapterError::new(
ErrorCode::AppUnresponsive,
"Launched application exited before it presented a window",
)
.with_details(serde_json::json!({ "pid": pid, "complete": false }))
.with_suggestion("Check the application's own launch requirements, then retry.")
}
/// The grace covers the gap between an application reporting that it started
@ -95,6 +121,16 @@ fn grace_over(grace_ends_at: Option<Instant>, now: Instant) -> bool {
grace_ends_at.is_some_and(|end| now >= end)
}
/// The attaching path reports the display name, so the launching path has to
/// report the same thing for the same field. A window already carries it; the
/// requested identifier is the fallback when there is no window to ask.
#[cfg(target_os = "macos")]
fn launched_display_name(window: Option<&WindowInfo>, id: &str) -> String {
window
.map(|window| window.app.clone())
.unwrap_or_else(|| id.to_owned())
}
#[cfg(target_os = "macos")]
fn result_from_app(app: &AppInfo, window: Option<WindowInfo>) -> LaunchResult {
LaunchResult {
@ -112,7 +148,7 @@ fn result_from_launched(
id: &str,
) -> Result<LaunchResult, AdapterError> {
Ok(LaunchResult {
app: id.to_owned(),
app: launched_display_name(window.as_ref(), id),
pid: agent_desktop_core::ProcessId::try_from(launched.0)
.map_err(|_| AdapterError::internal("Launched process identifier is out of range"))?,
process_instance: Some(launched.1.clone()),

View file

@ -78,8 +78,15 @@ fn focus_state(
/// accessory view before its first document appears. Counting those makes the
/// only real window look ambiguous, and reports an application that has not
/// drawn anything yet as having several windows to choose between.
fn narrow_to_visible(windows: &mut Vec<WindowInfo>) {
windows.retain(|window| window.state.visible == Some(true));
///
/// A minimized window is the user's window and is kept: it is offscreen for a
/// reason the caller cares about, unlike a panel that was never meant to be
/// seen. Dropping it would report an application as having no window while its
/// window sits in the Dock.
fn narrow_to_real_windows(windows: &mut Vec<WindowInfo>) {
windows.retain(|window| {
window.state.visible == Some(true) || window.state.minimized == Some(true)
});
}
pub(crate) fn exact_window_for_pid_until(
@ -96,7 +103,7 @@ pub(crate) fn exact_window_for_pid_until(
crate::system::process_identity::matches_instance(owner_pid, instance)
},
)?;
narrow_to_visible(&mut windows);
narrow_to_real_windows(&mut windows);
if windows.len() == 1 {
return Ok(windows.remove(0));
}

View file

@ -174,3 +174,43 @@ fn record(app_name: &str, pid: i32, title: &str, window_number: i64) -> WindowRe
process_instance: Some(format!("instance-{pid}")),
}
}
fn window_with_state(id: &str, visible: Option<bool>, minimized: Option<bool>) -> WindowInfo {
WindowInfo {
id: id.to_owned(),
title: "w".into(),
app: "Fixture".into(),
pid: agent_desktop_core::ProcessId::new(42),
process_instance: Some("fixture-42".into()),
bounds: None,
state: agent_desktop_core::WindowState {
is_focused: false,
minimized,
visible,
},
}
}
#[test]
fn narrowing_drops_bookkeeping_panels_but_keeps_a_minimized_window() {
let mut windows = vec![
window_with_state("panel", Some(false), Some(false)),
window_with_state("never-drawn", Some(false), None),
window_with_state("minimized", Some(false), Some(true)),
window_with_state("onscreen", Some(true), Some(false)),
];
narrow_to_real_windows(&mut windows);
let kept = windows.iter().map(|w| w.id.as_str()).collect::<Vec<_>>();
assert_eq!(kept, vec!["minimized", "onscreen"]);
}
#[test]
fn narrowing_leaves_nothing_when_an_application_has_only_panels() {
let mut windows = vec![window_with_state("panel", Some(false), Some(false))];
narrow_to_real_windows(&mut windows);
assert!(windows.is_empty());
}

View file

@ -4,7 +4,7 @@ Every command returns structured JSON:
```json
{
"version": "2.2",
"version": "2.3",
"ok": true,
"command": "click",
"data": { "action": "click" }
@ -15,7 +15,7 @@ Errors include machine-readable codes and recovery hints:
```json
{
"version": "2.2",
"version": "2.3",
"ok": false,
"command": "click",
"error": {

View file

@ -102,8 +102,8 @@ Use **progressive skeleton traversal** as the default approach. It reduces token
Every command returns a JSON envelope on stdout:
**Success:** `{ "version": "2.2", "ok": true, "command": "snapshot", "data": { ... } }`
**Error:** `{ "version": "2.2", "ok": false, "command": "click", "error": { "code": "STALE_REF", "message": "...", "suggestion": "..." } }`
**Success:** `{ "version": "2.3", "ok": true, "command": "snapshot", "data": { ... } }`
**Error:** `{ "version": "2.3", "ok": false, "command": "click", "error": { "code": "STALE_REF", "message": "...", "suggestion": "..." } }`
The `error` object may also carry an optional `details` object (e.g. the actionability report on an actionability failure, candidate summaries on `AMBIGUOUS_TARGET`, or the last observed state on a `wait` `TIMEOUT`). Parse errors leniently — `details` and future fields are additive, so do not reject responses with unknown keys.

View file

@ -37,7 +37,7 @@ agent-desktop snapshot --root @e12 --snapshot <snapshot_id> -i
**Output structure:**
```json
{
"version": "2.2",
"version": "2.3",
"ok": true,
"command": "snapshot",
"data": {

View file

@ -35,9 +35,11 @@ The process starting and the app presenting a window are separate outcomes, so t
A launch waits only for the windows the launch itself causes. It polls until the app reports that it finished starting up, plus a short grace for the first window to reach the window server. Most apps therefore return their window in one step. An app that opens its first window only when brought forward — any document-based app — returns without one instead of waiting out `--timeout`.
A launch that finds its process gone before any window appears fails with `APP_UNRESPONSIVE` rather than reporting a windowless success.
When you need the window:
- `--activate` asks the app to present one and waits for it. This brings the app forward, so it is not headless.
- `--activate` asks the app to present one and waits for it up to `--timeout`, because activation is what causes the window. This brings the app forward, so it is not headless. Pair it with a small `--timeout` for an app that may have no window at all.
- `wait --event window-opened` waits on your terms after you trigger the window some other way.
Windowless, menu-bar-only, and background apps simply report no `window`; use `list-apps` to observe those processes and read their `presentation`. `--no-attach` rejects an already-running app with `ACTION_FAILED` and starts a fresh instance.
@ -360,7 +362,7 @@ Each entry may include `"session": "id"` beside `command` and `args`. If omitted
**Per-entry failure shape:**
```json
{
"version": "2.2",
"version": "2.3",
"ok": false,
"command": "click",
"error": {