backend: a supplemented album remembers what it is between runs

A release Deezer's regional catalog omits is injected under a synthetic id,
and the only thing that can turn that id back into a release group is
`mb_catalog` / `apple_catalog` — two in-memory maps written only while an
artist page is built. Reaching such a record any other way fell through to
`api.deezer.com/album/{synthetic}`, which cannot resolve it, and the user was
told "album lookup failed".

Measured: King Gizzard's "The Silver Cord" (6660585962991362) failed cold in
237ms while the real Deezer single beside it (489399625) returned 14 tracks.
Three routes hit it — the sidebar's recent places, a replayed artist page
clicked before the fresh one lands, and the registries' 1000-entry clear.
That last one is why it looked intermittent.

Both registries now persist beside the MusicBrainz and Wikipedia bodies in
net-cache-v1, and the lookup reads memory, then disk, then Deezer. After the
fix the same cold lookup returns the record in 150ms.

The two structs deserialize as each other's type only if their required
fields happen to match, and they do not, so one entry can never be read as
the other kind.
This commit is contained in:
KeelrLeek committed 2026-07-29 18:45:23 -05:00
1 parent 0ea4da7146
commit 4f485b6f9a
1 file changed
+79 -9
+79 -9
View File
@@ -875,7 +875,7 @@ const HERO_SCORE_RECIPE: &str = "bing-identity-visual-v1";
/// A MusicBrainz studio album injected when Deezer's regional catalog omits the
/// release entirely. The synthetic numeric id keeps the existing frontend API
/// intact; this map routes an opened card back to its release group.
#[derive(Debug, Clone)]
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
struct MbCatalogAlbum {
release_group_id: String,
/// The ARTIST's MusicBrainz id, not the release group's.
@@ -1025,7 +1025,7 @@ struct DeezerReleaseEvidence {
/// card can fetch its tracklist from iTunes. The Apple analogue of
/// [`MbCatalogAlbum`]: Apple releases carry large synthetic ids that Deezer's
/// `/album/{id}` endpoint cannot resolve, so `catalog_album` routes them here.
#[derive(Debug, Clone)]
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
struct AppleCatalogAlbum {
collection_id: u64,
/// The DEEZER artist id this supplement was fetched for.
@@ -1339,6 +1339,54 @@ impl MetadataClient {
Some(dir.join(format!("{:016x}.json", hasher.finish())))
}
/// Where a supplemented album's identity is kept between runs.
///
/// A release Deezer's regional catalog omits is injected with a SYNTHETIC
/// id, and the only thing that can turn that id back into a release group
/// (or an Apple collection) is [`Self::mb_catalog`] / [`Self::apple_catalog`]
/// — two in-memory maps written when an artist page is built. Anything that
/// reaches such an album without having built that page in THIS process
/// falls through to `https://api.deezer.com/album/{synthetic}`, which
/// cannot resolve, and the user is told "album lookup failed".
///
/// Measured: King Gizzard's "The Silver Cord" (id 6660585962991362) fails
/// cold in 237ms while the real Deezer single beside it (489399625) returns
/// 14 tracks. Reaching it from the sidebar's recent places, from a replayed
/// artist page, or after the registries hit their 1000-entry `clear()` all
/// produce exactly that. Persisting the mapping makes the id resolvable
/// again without another artist-page build.
fn supplement_cache_path(&self, album_id: u64) -> Option<std::path::PathBuf> {
let dir = self.disk_cache_dir.as_ref()?;
Some(dir.join(format!("supplement-{album_id:016x}.json")))
}
/// Stable key for the entry above, so a filename collision cannot serve one
/// album's identity for another (same guard `read_disk_cache` already applies
/// to URLs).
fn supplement_cache_key(album_id: u64) -> String {
format!("soulstream://supplement/{album_id}")
}
/// Remember a supplemented album on disk as well as in memory.
fn persist_supplement<T: serde::Serialize>(&self, album_id: u64, entry: &T) {
let Some(path) = self.supplement_cache_path(album_id) else {
return;
};
let Ok(body) = serde_json::to_value(entry) else {
return;
};
write_disk_cache(&path, &Self::supplement_cache_key(album_id), &body);
}
/// Read one back. `None` when it was never registered, or is older than
/// [`DISK_CACHE_TTL`] — in which case opening the artist page registers it
/// again, exactly as before.
fn load_supplement<T: serde::de::DeserializeOwned>(&self, album_id: u64) -> Option<T> {
let path = self.supplement_cache_path(album_id)?;
let body = read_disk_cache(&path, &Self::supplement_cache_key(album_id))?;
serde_json::from_value(body).ok()
}
/// Enable verbose hero-candidate diagnostics. Intended for
/// `hero_source_probe`; it does not alter selection behavior.
pub fn with_hero_diagnostics(mut self) -> Self {
@@ -5190,17 +5238,19 @@ impl MetadataClient {
let cover = Some(format!(
"https://coverartarchive.org/release-group/{release_group_id}/front-500"
));
registry.insert(
id,
MbCatalogAlbum {
let entry = MbCatalogAlbum {
release_group_id: release_group_id.to_string(),
artist_mbid: artist_mbid.to_string(),
artist: artist.to_string(),
title: title.clone(),
release_date: release_date.clone(),
cover: cover.clone(),
},
);
};
// Disk as well as memory: this map is the ONLY way back from a
// synthetic id, and it dies with the process (and with the 1000-entry
// clear below). See `supplement_cache_path`.
self.persist_supplement(id, &entry);
registry.insert(id, entry);
albums.push(CatalogAlbum {
id,
title,
@@ -5524,10 +5574,28 @@ impl MetadataClient {
// never carried, now has play counts at all.
let mut deezer_artist_id: Option<u64> = None;
let mut artist_mbid: Option<String> = None;
let mut page = if let Some(album) = self.mb_catalog.lock().await.get(&album_id).cloned() {
// Memory first, then disk. The registries are process-lifetime maps
// written only while an artist page is built, so reaching a
// supplemented record any other way — the sidebar's recent places, a
// replayed page, or after the 1000-entry clear — used to fall through
// to Deezer with a synthetic id it cannot resolve, and report "album
// lookup failed". The disk copy makes the id resolvable on its own.
let mb_entry = match self.mb_catalog.lock().await.get(&album_id).cloned() {
Some(entry) => Some(entry),
None => self.load_supplement::<MbCatalogAlbum>(album_id),
};
let apple_entry = if mb_entry.is_some() {
None
} else {
match self.apple_catalog.lock().await.get(&album_id).cloned() {
Some(entry) => Some(entry),
None => self.load_supplement::<AppleCatalogAlbum>(album_id),
}
};
let mut page = if let Some(album) = mb_entry {
artist_mbid = Some(album.artist_mbid.clone());
self.musicbrainz_album(album_id, album).await?
} else if let Some(album) = self.apple_catalog.lock().await.get(&album_id).cloned() {
} else if let Some(album) = apple_entry {
// Apple has no MusicBrainz id of its own, but it does know whose
// page it was supplementing, so it reaches the scrobble table by
// the ordinary Deezer-id route.
@@ -6926,6 +6994,8 @@ impl MetadataClient {
registry.clear();
}
for (id, entry) in registered {
// Same reasoning as the MusicBrainz registry above.
self.persist_supplement(id, &entry);
registry.insert(id, entry);
}
}