import: the whole feature, and releases put back together

Everything the import grew over the last week, landing in one go — the
window, the picker, the audition pen, apply, and the review that stands
between them.

The fold is the new part. A release filed as a container of parts arrived
as N albums, because the scan groups audio by its immediate parent and the
container holds none of its own: Analord's eleven volumes were eleven
records, and six more releases split the same way. Records now fold into
the release above them, and apply is handed an explicit part list — never a
recursive walk, which is what let one stray file swallow 288 descendants
and duplicate 9 GB.

Where releases live is read off the collection rather than assumed, so a
genre-nested library moves the answer instead of breaking; below a dominant
majority at one depth the pass refuses entirely, because no folding beats
wrong folding. Three vetoes independent of depth catch a misread level.
Every fold is listed in the review with a way out.

research/import/fold-audit.mjs imports the shipping module, so the audited
code is the running code: 9 folds, 0 swallows across four library shapes,
and none of the 148 artist folders touched. It was that harness that killed
the tempting version of this rule — matching a part against its container
folded 40 artist folders, Mac Demarco's 8 albums into one, because album
folders carry the artist's name.

The regression test found a live bug on the way: parts number from 01 each,
so flattening two discs collided, and "same name, same size" read the
second file as already-imported and dropped it while still counting it.
That reuse now also requires that this run has not already written the
path.

Also here: the picker's accent crossfades between records instead of
cutting through the app's blue (ownership moved to the window, which
survives the per-record re-key), the fullscreen viewer wears the album's
accent instead of the one it left behind in the portal, Skip defers a
record without deciding it, and Keep mine is gone — with the cards picked
field by field it was Next under another name.
This commit is contained in:
KeelrLeek committed 2026-08-10 07:55:59 -05:00
1 parent 4b30c5e573
commit e7d0971de2
23 files changed
+6498 -258

No files matched your search

+203
View File
@@ -0,0 +1,203 @@
// Folding a release that was scanned as its parts.
//
// The scan groups audio by its IMMEDIATE parent folder, so a release filed as
// a container of parts —
//
// Aphex Twin/2005 - AFX - Analord, Volume 1-11/AFX - Analord 01/ 7 tracks
// /AFX - Analord 02/ 4 tracks
// … 11 volumes
//
// — arrives as eleven albums, and the container itself as none, because it
// holds no audio of its own. Six more of Kyle's records split the same way
// (Tago Mago, Drukqs, SAW II, 26 Mixes, Fishmans, Hella): 27 records that are
// really 7 releases.
//
// The rule and every guard in it were settled against the real 515-record scan
// BEFORE any of this shipped — `research/import/fold-audit.mjs` imports this
// module, so the audited code is the running code. Re-run it after any change
// here; it prints every fold and counts swallows under four library shapes.
//
// What the audit killed, and why the guards look like this:
//
// · Matching a part's name against its CONTAINER folded 40 of 148 artist
// folders — Mac Demarco's 8 albums into one — because album folders
// routinely carry the artist's name. Parts are compared to EACH OTHER.
// · A bare "depth 3 is a part" rule is only true of libraries shaped like
// his. The release level is read off the collection instead, so a
// genre-nested library moves the answer instead of breaking it.
// · Reading that level from an inconsistently nested collection merged 5
// records under the wrong parent. Below a dominant majority the pass
// refuses entirely: no folding beats wrong folding (see §0.1, 9 GB).
const SEP = "\\";
export const segs = (rel) => rel.split(SEP);
export const leaf = (rel) => segs(rel).at(-1);
/** Dropped before anything else looks at the list — a zip's `__MACOSX`
* resource fork otherwise reads as disc two. Mirrors ImportWindow's JUNK. */
const JUNK = /^(__MACOSX|\.DS_Store|@eaDir|System Volume Information)$/i;
/** `CD1`, `Disc 2`, `Side B`, `Part 3` — a part that says so in its name. */
const PART_NAME = /^(?:cd|disc|disk|side|part|vol(?:ume)?)[\s_.-]*\d+[a-z]?$/i;
const STOP = new Set(["the", "a", "an", "of", "and", "vol", "volume", "cd", "disc",
"disk", "part", "ep", "lp", "remastered", "deluxe", "edition", "records", "web",
"mp3", "flac", "kbps", "vinyl", "reissue"]);
const strip = (s) => s
.replace(/^[([]\d{4}[)\]]\s*-?\s*/, "").replace(/^\d{4}\s*-\s*/, "")
.replace(/[[(].*?[\])]/g, " ");
const nameTokens = (s) => new Set(
strip(s).toLowerCase().match(/[\p{L}\p{N}]+/gu)
?.filter((t) => t.length >= 3 && !STOP.has(t) && !/^\d+$/.test(t)) ?? []);
/** The folder name a folded record wears, and the year in it. The container's
* own name is the release; the parts' TAGS name the parts ("Analord 01"), so
* a tag majority over a folded record would title the whole set after its
* biggest volume. */
export const titleOf = (name) => strip(name).replace(/\s{2,}/g, " ").trim() || name;
export function yearOf(name) {
const m = name.match(/^(19|20)\d{2}(?=\s*[-–—]\s)/)
?? name.match(/[([]((?:19|20)\d{2})[)\]]/)
?? name.match(/\b((?:19|20)\d{2})\b/);
if (!m) return null;
const y = Number(m[0].replace(/\D/g, ""));
return y >= 1900 && y <= 2100 ? y : null;
}
const yearsIn = (a) => {
const out = [];
if (a.year) out.push(Number(a.year));
for (const t of a.tracks ?? []) if (t.year) out.push(Number(t.year));
return out.filter((y) => y >= 1900 && y <= 2100);
};
/** Reasons NOT to fold, each independent of the depth rule so a misread
* release level is caught rather than obeyed. Measured against the disaster
* case (every artist folder treated as a container): 69 of 73 rejected. */
const VETOES = [
function tooManyParts({ parts }) {
return parts.length > 30 ? `${parts.length} parts — reads as a discography` : null;
},
function yearSpread({ parts }) {
const ys = parts.flatMap(yearsIn);
if (ys.length < 2) return null;
const lo = Math.min(...ys), hi = Math.max(...ys);
return hi - lo > 1 ? `parts span ${hi - lo} years (${lo}–${hi})` : null;
},
function noCommonThread({ parts }) {
if (parts.length < 2) return null;
const names = parts.map((p) => leaf(p.rel));
if (names.every((n) => PART_NAME.test(strip(n).trim()))) return null;
const sets = names.map(nameTokens);
const shared = [...sets[0]].filter((t) => sets.every((s) => s.has(t)));
return shared.length ? null : "parts share no name between them";
},
];
/** Where releases live, read off the collection rather than assumed. */
export function releaseLevel(albums) {
const tally = new Map();
for (const a of albums) {
const d = segs(a.rel).length;
tally.set(d, (tally.get(d) ?? 0) + 1);
}
let level = 1, n = -1;
for (const [d, c] of [...tally].sort((x, y) => x[0] - y[0])) {
if (c > n) { n = c; level = d; }
}
return { level, share: albums.length ? n / albums.length : 0 };
}
/** Below this the collection has no single shape and nothing folds. */
export const CONFIDENCE = 0.7;
/**
* Fold nested parts into their release.
*
* @param {Array} input raw scan albums
* @param {Set<string>} [keepSeparate] container rels the user has un-folded
* @returns {{records, folds, vetoed, level, share, refused}}
* `records` is the list to use everywhere downstream; a folded one carries
* `parts` (sub-paths under its own rel, in order) so apply knows where the
* audio actually is without ever walking the tree.
*/
export function foldReleases(input, keepSeparate = new Set()) {
const albums = [], dropped = [];
for (const a of input) (JUNK.test(leaf(a.rel)) ? dropped : albums).push(a);
const { level, share } = releaseLevel(albums);
if (share < CONFIDENCE) {
return { records: albums, dropped, folds: [], vetoed: [], level, share, refused: true };
}
const byRel = new Map(albums.map((a) => [a.rel, a]));
const groups = new Map();
for (const a of albums) {
if (segs(a.rel).length <= level) continue;
const container = segs(a.rel).slice(0, level).join(SEP);
if (!groups.has(container)) groups.set(container, []);
groups.get(container).push(a);
}
const folds = [], vetoed = [], absorbed = new Set();
for (const [container, deeper] of groups) {
// A container holding loose audio beside its parts is a record too;
// its own files join the fold, first, as the unnamed part "".
const own = byRel.get(container) ?? null;
const parts = [...deeper].sort((x, y) => leaf(x.rel).localeCompare(
leaf(y.rel), undefined, { numeric: true, sensitivity: "base" }));
const all = own ? [own, ...parts] : parts;
const blocks = VETOES.map((v) => v({ container, parts: all })).filter(Boolean);
if (blocks.length) {
vetoed.push({ container, parts: all.length, reasons: blocks });
continue;
}
const entry = {
container,
name: titleOf(leaf(container)),
parts: all.length,
tracks: all.reduce((n, p) => n + (p.tracks?.length ?? 0), 0),
members: all.map((p) => leaf(p.rel)),
};
if (keepSeparate.has(container)) {
entry.off = true;
folds.push(entry);
continue;
}
folds.push(entry);
for (const p of all) absorbed.add(p.rel);
// Order is part order, then the scan's order within each part — the
// same order apply walks, because `trackTitles` is positional and a
// disagreement here would relabel every track past the first part.
const merged = {
rel: container,
artistFolder: segs(container)[0],
albumFolder: leaf(container),
depth: segs(container).length,
year: yearOf(leaf(container)) ?? all.map((p) => p.year).find(Boolean) ?? null,
folderArt: [...new Set(all.flatMap((p) => p.folderArt ?? []))],
tracks: all.flatMap((p) => p.tracks ?? []),
parts: all.map((p) => (p === own ? "" : leaf(p.rel))),
foldedFrom: all.length,
};
byRel.set(container, merged);
}
const records = [];
const emitted = new Set();
for (const a of albums) {
if (absorbed.has(a.rel)) {
const container = segs(a.rel).slice(0, level).join(SEP);
if (!emitted.has(container)) {
emitted.add(container);
records.push(byRel.get(container));
}
continue;
}
records.push(a);
}
return { records, dropped, folds, vetoed, level, share, refused: false };
}