diff --git a/ANDROID.md b/ANDROID.md index d3dc814..172fc67 100644 --- a/ANDROID.md +++ b/ANDROID.md @@ -857,6 +857,50 @@ with a `ClassNotFoundException`. `src-tauri/android/build.sh` exports it; after adding a plugin, touch `src-tauri/build.rs` too, so the app's build script re-runs and rewrites the wiring. +Every update also carries a **queue window** that nothing reads. `NowPlaying` +(Rust) and `NowPlayingArgs` (Kotlin) hold a `queue` of at most twenty-one +entries — the current item and ten either side, each with its id, title, +artist, album, duration and art key — plus `queueIndex`, where the current one +sits *within that window* rather than within the whole queue. +`src/mobile/media.svelte.js` fills both from `player.queue` and `player.qi`; +`MediaService` ignores them completely, so the card is byte for byte the card +it was. The reason is media3: `SimpleBasePlayer.getState()` wants a playlist +and an index into it, and the data that migration needs should have been +running on the phone for a while before anything depends on it. Ten either +side because one covers the card's own prev and next and nothing else, while a +queue *list* — which is what Auto asks for — wants a screenful plus a scroll +in both directions; and it is a cap, not a length, so a two-thousand-track +queue crosses the bridge as twenty-one entries exactly like a twenty-one-track +one does. Measured at two kilobytes of JSON for twenty-one short titles and +six with long ones and real Android cover paths, against the thirty to eighty +kilobytes of base64 cover that already ride the same call once per record — +and an update is a track change, a transport change or the thirty-second +top-up, never a position tick. + +Because it is on no screen, the walk reads it out of `logcat`: +`MediaSessionPlugin.update` logs one `media window n=… i=… art=… at=…` line +per update under the tag `SoulStream`, and the phone walk's `queue-window` +check holds that against `__ss.card` — the mirror `media.svelte.js` keeps of +what it last sent, beside xfernote's — field for field, through a next and a +previous. Field for field is deliberate: a key spelled snake_case where the +bridge wants camelCase arrives as nothing at all on both hops, with no error +anywhere, which is why the art-key count is compared and not just the size and +the index. + +Three things in that check are load-bearing and easy to undo by accident. It +arms at queue index **one** rather than wherever the walk left off, which buys +a row on either side (`prev()` seeks to zero at the top of a queue and +`next()` stops at the end), a track that has just started (`prev()` is a seek +again past five seconds), and the fact that while the queue index is at most +ten nothing is clamped off the front of the window — so the window's index IS +the queue index, which is what lets the logged index be compared against the +queue's directly. It requires each reading to be a FRESH one at both ends, the +push counter up and a new line in the log, because a `prev()` that lands back +on the row it started from reads exactly like a stale line otherwise. And it +FAILS rather than skips when it cannot be armed — a queue under three rows, or +repeat-one, under which `next()` seeks instead of stepping — with the reason +in the message. + ## What a screen reader says Every control on the phone is named by what it does, in the app's own @@ -1851,12 +1895,13 @@ read over real IPC; a cold artist page, its ⋯, the whole-discography page and its A–Z; the first-run preview with `setupWrites` at 0; eleven raw backend strings through `__ss.say`; and a refused transfer's sentence. -Then, after the step loop, seven things the mock cannot do at all: a +Then, after the step loop, eight things the mock cannot do at all: a force-stop and the paused restore that follows it (waited out past the thirteen-second FGS-kill window), the position restored, playback resumed, the system media session and its ownership, media keys, ten seconds in the -background with the position still advancing, and SoulStream's own mixer -underruns. +background with the position still advancing, SoulStream's own mixer +underruns, and the queue window that rides along with every card update and +that nothing on the card reads. One block in the phone walk pulls bytes, and only when the app is signed in: the trailing network block types `--query` into Search, streams a peer's copy diff --git a/crates/tauri-plugin-media-session/android/consumer-rules.pro b/crates/tauri-plugin-media-session/android/consumer-rules.pro index 02f0d3c..555c7b3 100644 --- a/crates/tauri-plugin-media-session/android/consumer-rules.pro +++ b/crates/tauri-plugin-media-session/android/consumer-rules.pro @@ -7,8 +7,9 @@ # FindClass), constructed through "(Landroid/app/Activity;)V", and its two # @Command methods — update and clear — are dispatched by NAME through # PluginHandle's commands map. NowPlayingArgs is filled by Jackson field by -# field. Rename any of it and the app still builds, installs and plays; the -# lock-screen card simply never appears. +# field, and QueueItemArgs — one per entry of the queue window it carries — +# the same way. Rename any of it and the app still builds, installs and +# plays; the lock-screen card simply never appears. # # The other half of this plugin is in the app-wide file: the card's buttons # come back as the plugin event "action", which the WebView subscribes to @@ -20,3 +21,4 @@ # them by name. Not repeated here: one place decides each of them. -keep class com.soulstream.mediasession.MediaSessionPlugin { *; } -keep class com.soulstream.mediasession.NowPlayingArgs { *; } +-keep class com.soulstream.mediasession.QueueItemArgs { *; } diff --git a/crates/tauri-plugin-media-session/android/src/main/java/com/soulstream/mediasession/MediaSessionPlugin.kt b/crates/tauri-plugin-media-session/android/src/main/java/com/soulstream/mediasession/MediaSessionPlugin.kt index 5968d6b..6c2f8cc 100644 --- a/crates/tauri-plugin-media-session/android/src/main/java/com/soulstream/mediasession/MediaSessionPlugin.kt +++ b/crates/tauri-plugin-media-session/android/src/main/java/com/soulstream/mediasession/MediaSessionPlugin.kt @@ -28,6 +28,32 @@ class NowPlayingArgs { var art: String? = null var hasNext: Boolean = true var hasPrev: Boolean = true + /** the queue around the current item — it and up to ten either side. + * NOTHING READS IT: MediaService ignores both of these on purpose, so + * the card is byte for byte the card it was. media3's + * SimpleBasePlayer.getState() wants a playlist and an index into it, + * and this is that data arriving a round before the migration does. + * Nullable because a non-null Kotlin property whose setter Jackson + * hands a null throws, and the only thing between here and the JS that + * builds it is two JSON hops. */ + var queue: List? = null + /** where the current item sits in `queue` — not in the whole queue; + * null when the window is empty */ + var queueIndex: Int? = null +} + +/** One entry of the queue window. Flat and small: it crosses the bridge on + * every update, so it holds what a media3 MediaItem is built from and no + * more. `artKey` names the cover the way NowPlayingArgs.artKey does; the + * bytes never ride along here. */ +@InvokeArg +class QueueItemArgs { + var id: String? = null + var title: String? = null + var artist: String? = null + var album: String? = null + var duration: Double? = null + var artKey: String? = null } /** @@ -52,6 +78,20 @@ class MediaSessionPlugin(private val activity: Activity) : Plugin(activity) { @Command fun update(invoke: Invoke) { val args = invoke.parseArgs(NowPlayingArgs::class.java) + /* The window is on no screen anywhere — MediaService ignores it and + the card cannot show it — so logcat is the only place on the + device it can be read back, and tests/mobile/smoke.mjs reads it + from exactly here. One line per update, and an update is a track + change, a transport change or the thirty-second top-up, never a + position tick. `at=` goes LAST because a title may hold anything, + spaces and `=` included. */ + val window = args.queue ?: emptyList() + android.util.Log.i( + "SoulStream", + "media window n=" + window.size + " i=" + (args.queueIndex ?: -1) + + " art=" + window.count { !it.artKey.isNullOrEmpty() } + + " at=" + (window.getOrNull(args.queueIndex ?: -1)?.title ?: ""), + ) ensureNotificationPermission() MediaHub.update(activity.applicationContext, args) invoke.resolve() diff --git a/crates/tauri-plugin-media-session/src/lib.rs b/crates/tauri-plugin-media-session/src/lib.rs index 1f55146..0aeefb5 100644 --- a/crates/tauri-plugin-media-session/src/lib.rs +++ b/crates/tauri-plugin-media-session/src/lib.rs @@ -41,12 +41,50 @@ pub struct NowPlaying { pub has_next: bool, #[serde(default = "yes")] pub has_prev: bool, + /// The queue around what is playing: the current item and up to ten + /// either side, rebuilt on every update. + /// + /// NOTHING READS IT YET, and that is the point — `MediaService` ignores + /// both of these fields, so the card is exactly the card it was. media3's + /// `SimpleBasePlayer.getState()` wants a playlist and an index into it, + /// and when that migration comes the data it needs should have been + /// running on the phone for a while rather than have been written the + /// same day. + /// + /// A CAP, not a length: a two-thousand-track queue crosses the bridge as + /// twenty-one entries exactly like a twenty-one-track one does. + #[serde(default)] + pub queue: Vec, + /// Where the current item sits IN `queue` — not in the whole queue. + /// `None` when the window is empty. + pub queue_index: Option, } fn yes() -> bool { true } +/// One entry of the queue window. Deliberately flat and small: this crosses +/// two hops of IPC on every update — the WebView to Rust, Rust to Kotlin — +/// so it carries what a media3 `MediaItem` is built from and nothing else. +/// `art_key` is the same kind of key `NowPlaying::art_key` is, the cover's +/// path, so whatever decodes a bitmap for the card can key the window's +/// covers off the same thing; no bytes ride along here. +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +#[serde(rename_all = "camelCase")] +pub struct QueueItem { + /// The track's own id as a string — a library track's is a number, a + /// catalog track's is `cat:`, a playlist row's is `pl`. NOT a + /// unique key: one queue may hold the same track twice, and a window + /// entry is identified by where it sits, not by this. + pub id: Option, + pub title: Option, + pub artist: Option, + pub album: Option, + pub duration: Option, + pub art_key: Option, +} + #[cfg(mobile)] pub struct MediaSession(tauri::plugin::PluginHandle); diff --git a/src/mobile/Mobile.svelte b/src/mobile/Mobile.svelte index efd6d61..a214f0d 100644 --- a/src/mobile/Mobile.svelte +++ b/src/mobile/Mobile.svelte @@ -47,7 +47,7 @@ import { loadPrefs } from "./prefs.svelte.js"; import { setup, loadSetup, maybeOpenFirstRun, openSetup, finishSetup, setupState } from "./setup.svelte.js"; import { catalog, artistKey } from "./catalog.svelte.js"; - import { bindMedia, syncMedia } from "./media.svelte.js"; + import { bindMedia, syncMedia, mediacard } from "./media.svelte.js"; import { slsk, bindSlsk, connectStored, refreshRooms, unreadTotal } from "./slsk.svelte.js"; import { net, bindNet } from "./net.svelte.js"; import { resolver, streamSong, streamCatalogAlbum } from "./resolve.svelte.js"; @@ -323,7 +323,7 @@ requestAnimationFrame(() => requestAnimationFrame(() => { clearTimeout(sealTimer); sealBoot(); })); /* the probe handle — the desktop has __soulstream for the same reason: tests/mobile/smoke.mjs drives the real app through it */ - window.__ss = { nav, top, go, push, back, openSheet, closeSheet, openHalf, closeHalf, player, lib, art, catalog, home, loadHome, lyr, loadLyrics, slsk, boot, net, resolver, streamSong, streamCatalogAlbum, sources, api, transfers, xfn: xfernote, pls, notice, importer, artTally, imp: { startFolder, startFile }, setup, openSetup, setupState, wifi: { downloadsAllowed, cellular }, next, prev, toggle, seekSecs, playQueueAt, patch, contacts, loadContacts, say, refreshRooms }; + window.__ss = { nav, top, go, push, back, openSheet, closeSheet, openHalf, closeHalf, player, lib, art, catalog, home, loadHome, lyr, loadLyrics, slsk, boot, net, resolver, streamSong, streamCatalogAlbum, sources, api, transfers, xfn: xfernote, card: mediacard, pls, notice, importer, artTally, imp: { startFolder, startFile }, setup, openSetup, setupState, wifi: { downloadsAllowed, cellular }, next, prev, toggle, seekSecs, playQueueAt, patch, contacts, loadContacts, say, refreshRooms }; return () => { if (unlisten) unlisten(); }; }); diff --git a/src/mobile/media.svelte.js b/src/mobile/media.svelte.js index 433ceb9..cf89493 100644 --- a/src/mobile/media.svelte.js +++ b/src/mobile/media.svelte.js @@ -8,6 +8,54 @@ import { invoke, addPluginListener } from "@tauri-apps/api/core"; import { player, hooks, toggle, next, prev, seekSecs } from "./player.svelte.js"; import { art } from "./art.svelte.js"; +/* THE QUEUE WINDOW. Every update carries a bounded slice of the queue — + the current item and QUEUE_WINDOW either side — because media3's + SimpleBasePlayer.getState() wants a playlist and an index into it, and + the migration to it should find this data already running rather than + written the same day. Nothing reads it yet: MediaService ignores both + fields and the card is exactly the card it was. + + TEN either side, twenty-one entries at most. One would cover the card's + own prev and next and nothing else; a queue LIST — which is what Auto + asks for — wants a screenful plus a scroll in either direction, and a + car's list pages about five rows, so ten is two of those each way. It is + a CAP, not a length: a two-thousand-track queue crosses the bridge as + twenty-one entries exactly like a twenty-one-track one does. Measured at + two kilobytes of JSON for twenty-one short titles and six with long ones + and real Android cover paths — against the thirty to eighty kilobytes of + base64 cover that already ride this same call once per record, and an + update is a track change, a transport change or the thirty-second top-up, + never a position tick. */ +const QUEUE_WINDOW = 10; + +/** A queue entry as the window carries it: what a media3 MediaItem is built + * from, flat, with the cover NAMED rather than carried. The artist falls + * back to the record's the way the bar and the sheet's rows do. */ +function windowItem(e) { + return { + id: e?.t?.id != null ? String(e.t.id) : null, + title: e?.t?.title || "", + artist: e?.t?.artist || e?.album?.artist || "", + album: e?.album?.title || "", + duration: e?.t?.duration_secs ?? null, + artKey: e?.album?.art_path || null, + }; +} + +/* What the card was last told about the queue around it. A PLAIN object, + not $state — nothing renders it, and push() must never make the bar's own + effects re-run — exactly as xfernote keeps its own mirror of the shade. + The walk reads it through __ss.card and holds it against what the plugin + logs on the far side of the bridge: a lock-screen card is not a thing a + page can be asked about, and this window is not even on the card. */ +export const mediacard = { + updates: 0, + window: 0, + index: -1, + art: 0, + at: "", +}; + let lastArtKey = null; let timer = null; let bound = false; @@ -17,6 +65,10 @@ async function push() { const t = player.track; if (!t) { lastArtKey = null; + mediacard.window = 0; + mediacard.index = -1; + mediacard.art = 0; + mediacard.at = ""; await invoke("plugin:media-session|clear").catch(() => {}); return; } @@ -34,11 +86,27 @@ async function push() { hasNext: player.qi + 1 < player.queue.length || player.repeat > 0, hasPrev: true, }; + /* the window, clamped at both ends. `at` is recomputed rather than + assumed to be QUEUE_WINDOW: at the top of the queue the slice starts + at 0 and the current item is the first entry, not the eleventh. */ + const lo = Math.max(0, player.qi - QUEUE_WINDOW); + const win = player.qi < 0 ? [] : player.queue.slice(lo, player.qi + QUEUE_WINDOW + 1); + const at = player.qi - lo; + state.queue = win.map(windowItem); + state.queueIndex = at >= 0 && at < win.length ? at : null; /* the cover's bytes ride along once per record, and only once they exist */ if (key && key !== lastArtKey && url && url.startsWith("data:image")) { state.art = url.slice(url.indexOf(",") + 1); lastArtKey = key; } + /* what was SENT, recorded before the call and whether or not it lands — + a desktop build and the mock both reject it, and neither is a reason + to forget what the window looked like */ + mediacard.updates++; + mediacard.window = state.queue.length; + mediacard.index = state.queueIndex ?? -1; + mediacard.art = state.queue.filter((q) => !!q.artKey).length; + mediacard.at = state.queueIndex == null ? "" : (state.queue[state.queueIndex].title || ""); await invoke("plugin:media-session|update", { state }).catch(() => {}); } diff --git a/tests/mobile/smoke.mjs b/tests/mobile/smoke.mjs index d44a3cd..39263bd 100644 --- a/tests/mobile/smoke.mjs +++ b/tests/mobile/smoke.mjs @@ -2485,6 +2485,23 @@ function mediaSessionState() { } catch { return { owner: "", state: "" }; } } +/* The queue window the phone was last handed, as MediaSessionPlugin logged + * it. Round forty-six put a bounded slice of the queue into every card + * update — the current item and ten either side — and left MediaService + * ignoring all of it, so there is nothing on any screen to read it back + * from: logcat is the only place on the device it exists, one line per + * update, and the LAST line is the state now. `at=` is last in the line + * because a title may hold anything, spaces and `=` included. */ +function mediaWindow() { + try { + const out = adb("logcat", "-d", "-s", "SoulStream:D").toString(); + const lines = out.split(/\r?\n/).filter((l) => l.includes("media window ")); + const m = (lines[lines.length - 1] || "").match(/media window n=(\d+) i=(-?\d+) art=(\d+) at=(.*)$/); + if (!m) return { n: -1, i: -1, art: -1, at: "", lines: lines.length }; + return { n: +m[1], i: +m[2], art: +m[3], at: m[4].trim(), lines: lines.length }; + } catch (e) { return { n: -1, i: -1, art: -1, at: "ERR " + e.message, lines: 0 }; } +} + /* ── the phone, turned over: the two settings that do it ─────────────── * `settings get system accelerometer_rotation` and `user_rotation` are the * only two things on the device this walk ever writes, and both are READ @@ -3003,6 +3020,90 @@ async function main() { if (!keysOk) problems.push("EXPECT media-keys " + JSON.stringify({ keyPaused, keyPlaying })); console.log((keysOk ? " ok " : " FAIL ") + "media-keys".padEnd(16) + " " + JSON.stringify({ paused: keyPaused?.status, playing: keyPlaying?.status })); + /* ── the queue window, across the bridge ─────────────────── + Round forty-six put a bounded window of the queue into every + card update — the current item and ten either side, each with + its id, title, artist, album, duration and art key, plus where + the current one sits among them — and left MediaService + ignoring every bit of it, because media3's + SimpleBasePlayer.getState() will want a playlist and an index, + and that data should be a round old before anything depends on + it. Which means every check above this one is blind to it: a + window that arrived empty, an index off by one, or a field the + bridge dropped because a key was spelled the wrong way would + each look exactly like success. + + So: what the phone SENT — __ss.card, the mirror + media.svelte.js keeps beside xfernote's — against what the + plugin LOGGED on the far side, field for field, through a next + and a previous. Field for field is the point: a camelCase key + spelled snake_case arrives as nothing at all on both hops of + this bridge, with no error anywhere, so `art` (the only + camelCase field an entry has) is compared as well as the size + and the index. + + ARMED TO A KNOWN ROW, index one, rather than to wherever the + walk left off. Three things fall out of that and every one of + them is load-bearing: there is a row on either side, so next() + steps instead of stopping at the end and prev() steps instead + of seeking to zero at the top; the track starts at zero, so + prev() is still a step two and a half seconds later rather + than the seek it becomes past five, and no track ends + underneath the three readings; and while the queue index is + under ten the window's own index IS the queue index, because + the slice starts at zero and nothing is clamped off the front + — which is what makes `got.i` comparable to `base` below + without recomputing the arithmetic here. Nothing is added, + removed or reordered: this is the queue the walk has been + playing all along, stepped one row forward and one row back + inside itself. + + Not skipped when it cannot be armed: a queue shorter than + three rows, or repeat-one — under which next() seeks to zero + instead of stepping — FAILS, with `armed` in the message + saying which. A check that passes by not running is the thing + this walk is most careful never to be. */ + const CARD = `(function () { var s = window.__ss; if (!s || !s.card || !s.player) return null; var c = s.card, p = s.player; return { u: c.updates, n: c.window, i: c.index, art: c.art, at: c.at, qi: p.qi, title: p.track ? p.track.title : '' }; })()`; + const armed = await cdp.evaluate(`(function () { var p = window.__ss && __ss.player; return p ? { n: p.queue.length, qi: p.qi, rep: p.repeat } : null; })()`); + const base = 1; + const room = (armed?.n ?? 0) >= 3 && armed.rep !== 2; + const wins = []; + if (room) { + await cdp.evaluate(`__ss.playQueueAt(${base})`); + await sleep(2500); + wins.push(["here", await cdp.evaluate(CARD), mediaWindow()]); + await cdp.evaluate(`__ss.next()`); + await sleep(2500); + wins.push(["next", await cdp.evaluate(CARD), mediaWindow()]); + await cdp.evaluate(`__ss.prev()`); + await sleep(2500); + wins.push(["prev", await cdp.evaluate(CARD), mediaWindow()]); + } + /* each reading: the window SENT is a real one — at least two + entries, the index inside it, and the entry AT the index is the + track on the bar, which is what an off-by-one fails even when + the counts agree — and what the plugin LOGGED is that same + window, field for field. */ + const sane = ([, sent, got]) => !!sent && sent.n >= 2 && sent.i >= 0 && sent.i < sent.n + && !!sent.at && sent.at === sent.title + && got.n === sent.n && got.i === sent.i && got.art === sent.art + && got.at === String(sent.at).trim(); + const winOk = room && wins.length === 3 && wins.every(sane) + /* and it MOVED. The queue index steps forward and back; the + LOGGED window index steps with it, which is the assertion + an off-by-one or a frozen index dies on. And each reading + is a FRESH one at both ends — the phone pushed again + (`u`) and the plugin logged again (`lines`) — because a + prev() that lands back on the row it started from reads + identically to a stale line otherwise. */ + && wins[0][1].qi === base && wins[1][1].qi === base + 1 && wins[2][1].qi === base + && wins[0][2].i === base && wins[1][2].i === base + 1 && wins[2][2].i === base + && wins[1][1].u > wins[0][1].u && wins[2][1].u > wins[1][1].u + && wins[1][2].lines > wins[0][2].lines && wins[2][2].lines > wins[1][2].lines; + if (!winOk) problems.push("EXPECT queue-window " + JSON.stringify({ armed, base, wins })); + console.log((winOk ? " ok " : " FAIL ") + "queue-window".padEnd(16) + " " + + JSON.stringify({ armed, seen: wins.map(([k, sent, got]) => [k, sent && sent.qi, got.n, got.i, got.art, got.at]) })); + const bgBefore = await cdp.evaluate(PLAYER); const posBefore = bgBefore?.pos ?? 0; cdp.close();