forked from Keelrleek/SoulStream
hero: tell a square press crop from a sleeve by hashing the artist's releases
The aspect floor rejected everything below 1.15 because a square raster is usually album art, and album art clears every other gate. That proxy also threw away square PRESS photography -- Nikki Nair's entire usable corpus is 1:1 crops from Rinse FM, Corsica Studios and 529 Atlanta, so he had no route to a hero at all. Shape cannot separate those two cases, but the catalogue can: a sleeve IS one of the artist's covers. A square candidate is now perceptually hashed against up to 25 of the artist's own releases and admitted only when it matches none, reusing the dhash and distance threshold already used for cover resolution. With no hashes available -- no Deezer id, or the lookup failed -- it falls back to the old blanket rejection rather than guessing. The covers are fetched alongside Bing discovery rather than after it, at 250px and mostly from the shared image cache, so it costs no wall clock next to the multi-megabyte hero candidates. The six-second seed path passes no hashes: it exists to paint something safe quickly, and a square candidate can wait for the exhaustive pass to adjudicate it. Nikki Nair goes from BANNER NONE to 2 -- a withguitars promo shot and a 529 Atlanta press photo, verified rendering correctly. Camping in Alaska is unchanged with its three banners, so the guard that this gate was originally added for still holds: its square "lost and found" sleeve stays out. Tests cover the fork directly: a square matching a release is rejected, a square matching none is admitted, a square with no catalogue knowledge is rejected, portrait is admitted, and a portrait too narrow to crop is not.
This commit is contained in:
1 parent
d79fe2f28a
commit
9cd03a6ecc
1 file changed
+180
-16
+180
-16
@@ -574,6 +574,9 @@ const HERO_PORTRAIT_MAX_ASPECT: f64 = 0.92;
|
|||||||
/// A portrait source is cropped to full-width x (width*9/16), so its width IS
|
/// A portrait source is cropped to full-width x (width*9/16), so its width IS
|
||||||
/// the resulting hero width. Keep enough pixels that the crop is still a hero.
|
/// the resulting hero width. Keep enough pixels that the crop is still a hero.
|
||||||
const HERO_PORTRAIT_MIN_WIDTH: u32 = 1000;
|
const HERO_PORTRAIT_MIN_WIDTH: u32 = 1000;
|
||||||
|
/// How many of the artist's releases to perceptually hash when deciding whether
|
||||||
|
/// a square candidate is one of their sleeves.
|
||||||
|
const HERO_COVER_HASH_LIMIT: usize = 25;
|
||||||
const HERO_SEARCH_WAVE_CONCURRENCY: usize = 3;
|
const HERO_SEARCH_WAVE_CONCURRENCY: usize = 3;
|
||||||
/// A Bing image page with at most this many cards is a degraded answer, not a
|
/// A Bing image page with at most this many cards is a degraded answer, not a
|
||||||
/// small result set. Bing's genuine pages run 12-100 cards; the degraded shape
|
/// small result set. Bing's genuine pages run 12-100 cards; the degraded shape
|
||||||
@@ -1632,12 +1635,21 @@ impl MetadataClient {
|
|||||||
) -> Vec<String> {
|
) -> Vec<String> {
|
||||||
let started = Instant::now();
|
let started = Instant::now();
|
||||||
let discovery_started = Instant::now();
|
let discovery_started = Instant::now();
|
||||||
let (discovery, linked_people) = tokio::join!(
|
// Hashed alongside discovery, not after it: the covers are 250px and
|
||||||
|
// mostly cache hits, so this costs no wall clock, and without them a
|
||||||
|
// square press photo cannot be told from a sleeve and is rejected.
|
||||||
|
let (discovery, linked_people, cover_dhashes) = tokio::join!(
|
||||||
tokio::time::timeout(
|
tokio::time::timeout(
|
||||||
Duration::from_secs(20),
|
Duration::from_secs(20),
|
||||||
self.bing_image_candidates(artist, policy),
|
self.bing_image_candidates(artist, policy),
|
||||||
),
|
),
|
||||||
self.bing_linked_people(artist, deezer_id, policy),
|
self.bing_linked_people(artist, deezer_id, policy),
|
||||||
|
async {
|
||||||
|
match deezer_id {
|
||||||
|
Some(id) => self.artist_cover_dhashes(id).await,
|
||||||
|
None => Vec::new(),
|
||||||
|
}
|
||||||
|
},
|
||||||
);
|
);
|
||||||
let Ok(candidates) = discovery else {
|
let Ok(candidates) = discovery else {
|
||||||
if self.hero_diagnostics {
|
if self.hero_diagnostics {
|
||||||
@@ -1654,7 +1666,9 @@ impl MetadataClient {
|
|||||||
filter_bing_candidates_for_policy(candidates, policy, artist, &linked_people);
|
filter_bing_candidates_for_policy(candidates, policy, artist, &linked_people);
|
||||||
let candidate_count = candidates.len();
|
let candidate_count = candidates.len();
|
||||||
let scoring_started = Instant::now();
|
let scoring_started = Instant::now();
|
||||||
let scored = self.score_bing_hero_candidates(candidates, artist).await;
|
let scored = self
|
||||||
|
.score_bing_hero_candidates(candidates, artist, &cover_dhashes)
|
||||||
|
.await;
|
||||||
let scoring_ms = scoring_started.elapsed().as_millis();
|
let scoring_ms = scoring_started.elapsed().as_millis();
|
||||||
let scored_count = scored.len();
|
let scored_count = scored.len();
|
||||||
let (selected, duplicates) = select_scored_hero_urls(scored);
|
let (selected, duplicates) = select_scored_hero_urls(scored);
|
||||||
@@ -1692,7 +1706,11 @@ impl MetadataClient {
|
|||||||
);
|
);
|
||||||
let candidates =
|
let candidates =
|
||||||
filter_bing_candidates_for_policy(candidates, policy, artist, &linked_people);
|
filter_bing_candidates_for_policy(candidates, policy, artist, &linked_people);
|
||||||
self.first_scored_strong_hero_seed(candidates, artist).await
|
// No cover hashes on the seed path: it exists to paint something
|
||||||
|
// safe within six seconds, and a square candidate simply waits for
|
||||||
|
// the exhaustive pass to adjudicate it against the catalogue.
|
||||||
|
self.first_scored_strong_hero_seed(candidates, artist, &[])
|
||||||
|
.await
|
||||||
})
|
})
|
||||||
.await
|
.await
|
||||||
.ok()
|
.ok()
|
||||||
@@ -1708,13 +1726,14 @@ impl MetadataClient {
|
|||||||
&self,
|
&self,
|
||||||
candidates: Vec<BingImageCandidate>,
|
candidates: Vec<BingImageCandidate>,
|
||||||
artist: &str,
|
artist: &str,
|
||||||
|
cover_dhashes: &[u64],
|
||||||
) -> Option<String> {
|
) -> Option<String> {
|
||||||
let mut scoring = stream::iter(
|
let mut scoring = stream::iter(
|
||||||
candidates
|
candidates
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.filter(|candidate| candidate.identity == BingIdentityConfidence::Strong)
|
.filter(|candidate| candidate.identity == BingIdentityConfidence::Strong)
|
||||||
.map(|candidate| async move {
|
.map(|candidate| async move {
|
||||||
self.hero_candidate_score(&candidate, artist)
|
self.hero_candidate_score(&candidate, artist, cover_dhashes)
|
||||||
.await
|
.await
|
||||||
.map(|_| candidate.url)
|
.map(|_| candidate.url)
|
||||||
}),
|
}),
|
||||||
@@ -1735,9 +1754,12 @@ impl MetadataClient {
|
|||||||
&self,
|
&self,
|
||||||
candidates: Vec<BingImageCandidate>,
|
candidates: Vec<BingImageCandidate>,
|
||||||
artist: &str,
|
artist: &str,
|
||||||
|
cover_dhashes: &[u64],
|
||||||
) -> Vec<(String, f64, HeroFingerprint, BingIdentityConfidence)> {
|
) -> Vec<(String, f64, HeroFingerprint, BingIdentityConfidence)> {
|
||||||
stream::iter(candidates.into_iter().map(|candidate| async move {
|
stream::iter(candidates.into_iter().map(|candidate| async move {
|
||||||
let visual = self.hero_candidate_score(&candidate, artist).await?;
|
let visual = self
|
||||||
|
.hero_candidate_score(&candidate, artist, cover_dhashes)
|
||||||
|
.await?;
|
||||||
let base = visual.score;
|
let base = visual.score;
|
||||||
// Relevance order is meaningful within a query, but the old global
|
// Relevance order is meaningful within a query, but the old global
|
||||||
// rank unfairly buried the first result from later portrait/DJ
|
// rank unfairly buried the first result from later portrait/DJ
|
||||||
@@ -2220,10 +2242,51 @@ impl MetadataClient {
|
|||||||
/// overlays, and video-thumbnail CDNs. These are priors rather than hard
|
/// overlays, and video-thumbnail CDNs. These are priors rather than hard
|
||||||
/// rejects so a sparse artist can still receive a usable hero.
|
/// rejects so a sparse artist can still receive a usable hero.
|
||||||
/// `None` = unusable (too small or undecodable).
|
/// `None` = unusable (too small or undecodable).
|
||||||
|
/// Perceptual hashes of the artist's OWN release artwork.
|
||||||
|
///
|
||||||
|
/// A square raster is a hero risk because it is usually a sleeve, but shape
|
||||||
|
/// cannot tell a sleeve from a square press crop. This can: a sleeve IS one
|
||||||
|
/// of these images. Deliberately cheap -- `cover_medium` is 250px, the
|
||||||
|
/// bytes come from the shared image cache, and hashing is microseconds --
|
||||||
|
/// so it costs far less than the multi-megabyte hero candidates alongside
|
||||||
|
/// it.
|
||||||
|
async fn artist_cover_dhashes(&self, deezer_id: u64) -> Vec<u64> {
|
||||||
|
let Some(body) = self
|
||||||
|
.deezer(&format!(
|
||||||
|
"https://api.deezer.com/artist/{deezer_id}/albums?limit=25"
|
||||||
|
))
|
||||||
|
.await
|
||||||
|
else {
|
||||||
|
return Vec::new();
|
||||||
|
};
|
||||||
|
let Some(items) = body.get("data").and_then(|data| data.as_array()) else {
|
||||||
|
return Vec::new();
|
||||||
|
};
|
||||||
|
let urls = items
|
||||||
|
.iter()
|
||||||
|
.filter_map(|item| {
|
||||||
|
item.get("cover_medium")
|
||||||
|
.and_then(|cover| cover.as_str())
|
||||||
|
.map(str::to_string)
|
||||||
|
})
|
||||||
|
.take(HERO_COVER_HASH_LIMIT)
|
||||||
|
.collect::<Vec<_>>();
|
||||||
|
let mut hashes = Vec::with_capacity(urls.len());
|
||||||
|
for url in urls {
|
||||||
|
if let Some(bytes) = self.get_image_shared(&url).await {
|
||||||
|
if let Some(image) = decode_hero_image_slice(bytes.as_slice()) {
|
||||||
|
hashes.push(dhash64(&image));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
hashes
|
||||||
|
}
|
||||||
|
|
||||||
async fn hero_candidate_score(
|
async fn hero_candidate_score(
|
||||||
&self,
|
&self,
|
||||||
candidate: &BingImageCandidate,
|
candidate: &BingImageCandidate,
|
||||||
artist: &str,
|
artist: &str,
|
||||||
|
cover_dhashes: &[u64],
|
||||||
) -> Option<HeroVisualScore> {
|
) -> Option<HeroVisualScore> {
|
||||||
let cache_key = format!(
|
let cache_key = format!(
|
||||||
"{}\u{1f}{}\u{1f}{}:{}\u{1f}{}:{}\u{1f}{}:{}",
|
"{}\u{1f}{}\u{1f}{}:{}\u{1f}{}:{}\u{1f}{}:{}",
|
||||||
@@ -2262,10 +2325,11 @@ impl MetadataClient {
|
|||||||
let candidate = candidate.clone();
|
let candidate = candidate.clone();
|
||||||
let artist = artist.to_string();
|
let artist = artist.to_string();
|
||||||
let diagnostics = self.hero_diagnostics;
|
let diagnostics = self.hero_diagnostics;
|
||||||
|
let cover_dhashes = cover_dhashes.to_vec();
|
||||||
let resolved = fetch
|
let resolved = fetch
|
||||||
.get_or_init(|| async move {
|
.get_or_init(|| async move {
|
||||||
tokio::task::spawn_blocking(move || {
|
tokio::task::spawn_blocking(move || {
|
||||||
score_bing_hero_image(bytes, &candidate, &artist, diagnostics)
|
score_bing_hero_image(bytes, &candidate, &artist, &cover_dhashes, diagnostics)
|
||||||
})
|
})
|
||||||
.await
|
.await
|
||||||
.ok()
|
.ok()
|
||||||
@@ -8827,6 +8891,7 @@ fn score_bing_hero_image(
|
|||||||
bytes: Arc<Vec<u8>>,
|
bytes: Arc<Vec<u8>>,
|
||||||
candidate: &BingImageCandidate,
|
candidate: &BingImageCandidate,
|
||||||
artist: &str,
|
artist: &str,
|
||||||
|
cover_dhashes: &[u64],
|
||||||
diagnostics: bool,
|
diagnostics: bool,
|
||||||
) -> Option<HeroVisualScore> {
|
) -> Option<HeroVisualScore> {
|
||||||
let url = &candidate.url;
|
let url = &candidate.url;
|
||||||
@@ -8924,9 +8989,36 @@ fn score_bing_hero_image(
|
|||||||
// 1024x1365) both pass identity and died here. Text-heavy posters and
|
// 1024x1365) both pass identity and died here. Text-heavy posters and
|
||||||
// flyers cannot sneak in this way — the text, flat-graphic and collage
|
// flyers cannot sneak in this way — the text, flat-graphic and collage
|
||||||
// gates all run BEFORE this one.
|
// gates all run BEFORE this one.
|
||||||
let usable_portrait =
|
//
|
||||||
ratio <= HERO_PORTRAIT_MAX_ASPECT && w >= HERO_PORTRAIT_MIN_WIDTH;
|
// SQUARE is admitted only when we can prove it is not a sleeve. Shape
|
||||||
if ratio < HERO_MIN_ASPECT && !usable_portrait && !trusted_ticketing_profile {
|
// cannot distinguish a 1:1 press crop (Nikki Nair's Rinse FM and Corsica
|
||||||
|
// Studios photographs) from release artwork, but the artist's own covers
|
||||||
|
// can: a sleeve IS one of them, so a perceptual-hash match against the
|
||||||
|
// catalogue answers the question the aspect ratio was only guessing at.
|
||||||
|
// With no cover hashes available -- no Deezer id, or the lookup failed --
|
||||||
|
// this falls back to the old blanket rejection rather than guessing.
|
||||||
|
let usable_portrait = ratio <= HERO_PORTRAIT_MAX_ASPECT && w >= HERO_PORTRAIT_MIN_WIDTH;
|
||||||
|
let usable_square = ratio > HERO_PORTRAIT_MAX_ASPECT
|
||||||
|
&& w >= HERO_PORTRAIT_MIN_WIDTH
|
||||||
|
&& !cover_dhashes.is_empty()
|
||||||
|
&& {
|
||||||
|
let hash = dhash64(&decoded);
|
||||||
|
let matches_release = cover_dhashes
|
||||||
|
.iter()
|
||||||
|
.any(|cover| (cover ^ hash).count_ones() <= COVER_SAME_ART_MAX_DISTANCE);
|
||||||
|
if matches_release && diagnostics {
|
||||||
|
eprintln!(
|
||||||
|
"HERO_REJECT\talbum-art\t{}x{}\tratio={:.2}\t{}",
|
||||||
|
w, h, ratio, url,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
!matches_release
|
||||||
|
};
|
||||||
|
if ratio < HERO_MIN_ASPECT
|
||||||
|
&& !usable_portrait
|
||||||
|
&& !usable_square
|
||||||
|
&& !trusted_ticketing_profile
|
||||||
|
{
|
||||||
if diagnostics {
|
if diagnostics {
|
||||||
eprintln!(
|
eprintln!(
|
||||||
"HERO_REJECT\taspect\t{}x{}\tratio={:.2}\t{}",
|
"HERO_REJECT\taspect\t{}x{}\tratio={:.2}\t{}",
|
||||||
@@ -9624,21 +9716,93 @@ mod tests {
|
|||||||
);
|
);
|
||||||
|
|
||||||
assert!(
|
assert!(
|
||||||
score_bing_hero_image(textured(1200, 1200), &candidate, "Camping In Alaska", false)
|
score_bing_hero_image(
|
||||||
.is_none(),
|
textured(1200, 1200),
|
||||||
"square release artwork must be rejected as a hero",
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_none(),
|
||||||
|
"square artwork must be rejected when the catalogue is unknown",
|
||||||
);
|
);
|
||||||
assert!(
|
assert!(
|
||||||
score_bing_hero_image(textured(1600, 900), &candidate, "Camping In Alaska", false)
|
score_bing_hero_image(
|
||||||
.is_some(),
|
textured(1600, 900),
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_some(),
|
||||||
"wide press photography must still score",
|
"wide press photography must still score",
|
||||||
);
|
);
|
||||||
// 4:3 press photography is common and sits above the floor.
|
// 4:3 press photography is common and sits above the floor.
|
||||||
assert!(
|
assert!(
|
||||||
score_bing_hero_image(textured(1600, 1200), &candidate, "Camping In Alaska", false)
|
score_bing_hero_image(
|
||||||
.is_some(),
|
textured(1600, 1200),
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_some(),
|
||||||
"4:3 press photography must still score",
|
"4:3 press photography must still score",
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// A square candidate is adjudicated against the artist's own releases:
|
||||||
|
// matching one is a sleeve, matching none is a press crop.
|
||||||
|
let square = textured(1200, 1200);
|
||||||
|
let square_hash = dhash64(
|
||||||
|
&decode_hero_image_slice(square.as_slice()).expect("decode square test raster"),
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
score_bing_hero_image(
|
||||||
|
square.clone(),
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[square_hash],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_none(),
|
||||||
|
"a square that IS one of the artist's covers must be rejected",
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
score_bing_hero_image(
|
||||||
|
square,
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[square_hash ^ u64::MAX],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_some(),
|
||||||
|
"a square press crop matching no release must be admitted",
|
||||||
|
);
|
||||||
|
|
||||||
|
// Portrait press photography is cropped to 16:9 around the face by the
|
||||||
|
// app, so it is admitted when it carries enough width to survive that.
|
||||||
|
assert!(
|
||||||
|
score_bing_hero_image(
|
||||||
|
textured(1600, 2000),
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_some(),
|
||||||
|
"portrait press photography must be admitted for face-aware cropping",
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
score_bing_hero_image(
|
||||||
|
textured(800, 1000),
|
||||||
|
&candidate,
|
||||||
|
"Camping In Alaska",
|
||||||
|
&[],
|
||||||
|
false
|
||||||
|
)
|
||||||
|
.is_none(),
|
||||||
|
"a portrait too narrow to crop into a hero must still be rejected",
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
Reference in new issue
Block a user