bookmeta: Cover::guessed says the file left the choice open — a Kindle
book's first image record, or an EPUB image picked because its name says
cover — so a caller with a register to ask can prefer that. A cover the
file names is never a guess. Cover::extension matches bibfetch's.
bibfetch: Open Library answers 403 to more than a hundred covers by ISBN
in five minutes from one address. That is now Error::Limited, not
Unreachable: the other form of the ISBN is not asked, the cover id still
is, since those are not limited, and the CLI exits 75. Covers asked by
ISBN are kept three seconds apart across the process (Covers::pace), which
stays under the limit however long a run goes. A deadline too long to add
to the clock no longer panics; it is no deadline.
bindings: fetch_cover and search_title for Python, and catena.Limited,
which fetch_identifier raises too when a register refuses for being asked
too often.
corpus-core: exit 75, a service refused for now.
A book that arrives as a file often has no identifier to ask about, only
a title and perhaps a name. `bibfetch search --title T [--author A]
[--limit N]` asks Open Library's index which books it has under those,
and Google Books only when Open Library has nothing, and prints one
record per candidate in the order the register gave them. From Rust,
`bibfetch::search(&agent, title, author, limit, attempts)`.
Each candidate is one edition. The index answers with works and, asked
for `editions.*`, names the edition of each that best matches; the ISBN,
publisher, language, year and cover come from that edition, the title,
authors, subjects and length from the work. The work's own ISBN list
mixes editions: the first War and Peace work lists four, while the
edition matched is a 1917 printing with none.
`id` is the first ISBN whose check digit agrees, in thirteen digits,
thirteen-digit entries taken first. A candidate with none is named by
the register's key: `olid` (OL…M for the edition, OL…W for the work when
no edition was named) or `googlebooks` (a volume id). Google's `OTHER`
shelfmarks are never read as ISBNs.
`cover_id` keeps the meaning it has in a lookup record: the edition's
own first cover, taken from `editions.cover_i` and only when the record
names that edition. The work's `cover_i` and `cover_edition_key` are not
asked for. A work with no edition named, and every Google Books
candidate, has none.
Open Library having nothing stands when Google Books is down; both down
is exit 69. Exit 0 a candidate, 1 none, 65 an empty title.
The retrying GET that lookups use is now `ask::get`, shared by both, and
no longer sleeps after its last attempt.
The cover is the first image reached by EPUB 3's cover-image property,
EPUB 2's <meta name="cover"> (by id, or by address where a converter wrote
one), the guide's cover page and the first <img> or SVG <image> on it, and
last any manifest image whose id or address says cover. A route that names
something unusable leaves a note and the next is asked.
Addresses are resolved against the document they are written in, with the
fragment off and percent-escapes undone; one that climbs out of the archive
or carries a scheme is not followed. Every member is read through a bound,
checked against the size the archive declares and enforced on the read
itself, so an understated entry stops at the cap too.
The media type is read off the bytes, now WebP as well, and a manifest that
disagrees is a note; bytes that are not an image are no cover. MAX_COVER
(16 MiB) is the cap for both formats, so a Kindle cover record larger than
that is left out as well.
Four formats a library of ebooks holds and this had no name for, so each
came back as text, as HTML or as unknown.
RTF is its opening group, after any byte-order mark and blank space.
Microsoft Reader is its signature at the front. A FictionBook is XML
whose root element is FictionBook, found by stepping over the
declaration, comments and doctype in the first four kilobytes. That walk
is over bytes rather than text, because a great many FictionBooks are
windows-1251 and would otherwise be called binary; other XML is still
HTML, as before.
A zip whose only real member is one FictionBook is zip-wrapped-fb2, the
same way a zipped PDF is zip-wrapped-pdf: handed to a converter as
FictionBook it is still a zip. A zip whose every real member is a JPEG,
PNG, GIF or WebP is cbz, once directories, ComicInfo.xml and desktop
clutter are passed over. One PDF or one readme among the pages and it is
not a comic. Only the names in the archive's directory are read, as for
every other zip.
The kind vocabulary grows by five: lit, fb2, rtf, cbz and
zip-wrapped-fb2.
The suite could read a cataloguing block and could ask a register, and
nothing showed the two together — which is the only way either is worth
acting on. A title page alone gives up a running head when the scan
starts late; an ISBN alone returns a confidently wrong book when the
catalogue has been carrying the wrong one for years.
The example's title comparison folds to letters and digits rather than
comparing strings. A register and a title page differ over case, over a
curly apostrophe against a straight one, and over whether the subtitle
counts as part of the title. An example that called those disagreements
would teach the wrong lesson, and did until it was fixed.
It runs on this library and finds both of the cases worth finding: a file
called Document5 by -ducati996 that both sources agree is Russell's
Problems of Philosophy, and a Pelikan whose stored ISBN belongs to a book
about sculpture.
Two things between a converted EPUB and its chapter titles.
The heading level taken as the chapter map was the shallowest present. Pandoc
writes the book's title as the only `#` and every real chapter as a `##`, so
the shallowest level had one heading in it and the whole book came back as a
single chapter. The dividing level is now the shallowest one used more than
once — a level used once is a title.
And an empty bracketed span survived the heading cleaner, because the pattern
wanted at least one character between the brackets. Pandoc writes an anchor
as exactly that: `## []{#link2H_INTR} Introduction`.
Gibbon's first volume, converted from the Project Gutenberg EPUB: 2 distinct
chapter titles before, 61 after, and none of them carrying an element id.
The harness compared these tools against the code they replaced — Python
imported from book-pipeline, podcast-pipeline and knowledge-mcp, and
JavaScript copied out of the podcast media server and indexer, each pinned to
a commit. All of it has since been deleted, so the live half cannot run and
the golden half pins answers against a source nobody can consult.
What was worth keeping was never the comparison. `KNOWN.md` argued, at
length, why a dozen of these tools deliberately do something a reader would
not expect: why pdfgutter measures the crop box throughout, why avplan finds
the encoder rather than reading one out of the environment, why mdscrub will
not manufacture a page boundary. Those arguments are about this code and
stand on their own, so they are now `docs/decisions.md`, rewritten to say
what each tool does and why rather than what it does differently.
Binding coverage is unaffected: the bindings carry 24 tests of their own, a
node smoke test against the built package, and a typecheck.
The README now lists the forms it handles with an example of each, says that
where the italics sit is what decides whether they are the title or the
journal, and sets out what it refuses and why — scripture, Migne, a
contributor's biography, a note that only thanks people, a page footer.
Recipes gains "What a book leans on": one bibliography read, then the same
over a shelf, counted per citing book so a work cited forty times in one
bibliography still counts once. Every command was run — thirty converted
books gave 14,481 references, 10,536 naming a work, in about a second.
`records.md` now points at the five tools whose record shapes live in their
own pages rather than in `corpus-core`.
And pandoc escapes an underscore, so the rule of them that stands in for a
repeated author arrives as `\_\_\_\_\_` and was not being recognised.
funkpower