bibfetch: an edition's facts from its own record, and a cover whole to its end

A lookup by ISBN took the year, publisher, language and page count from
Open Library's index, which answers for the work and gathers them from
every edition: Penguin's English War and Peace came back Spanish,
Norton's, and 1845. The index now gives only the title, the authors and
the subjects, and the edition record the rest (#6).

A cover must run to its end marker, FF D9 for a JPEG and IEND for a PNG,
so a body that only starts like an image is refused (#4). A relative
redirect whose query holds another address is resolved as relative (#5).

Closes #4, #5, #6.
This commit is contained in:
funkpower committed 2026-11-13 16:23:48 -05:00
1 parent e9f9e75c1d
commit 6b5d9c20ee
3 files changed
+148 -29

No files matched your search

+8 -4
View File
@@ -86,15 +86,18 @@ is a failure — nobody replied, and the identifier may be perfectly good.
|---|---|---|
| `doi` | Crossref | It assigns most scholarly DOIs |
| `arxiv` | the arXiv | Its own register of preprints |
| `isbn` | OpenLibrary's edition record, then its index, then Google Books | Books, without a key |
| `isbn` | OpenLibrary's index, then its edition record, then Google Books | Books, without a key |
One register is enough for a DOI or a preprint: the body that assigns the
identifier is the only one that has it. A book is different. An ISBN is
assigned by a publisher and recorded by whoever gets round to it, so an
answer that names the book but not who wrote it is common — OpenLibrary's
edition record names its authors by key rather than by name, and its index
names them. So the registers are asked in turn and the first complete answer
wins. An incomplete one is kept in case nothing better comes, and a register
names them. The index answers about the work, though, not the edition: the
publishers, languages, years and page counts it holds belong to every edition
at once, so only the title, the authors and the subjects are taken from it,
and the rest comes from the edition record for that ISBN. So the registers are
asked in turn and the first complete answer wins. An incomplete one is kept in case nothing better comes, and a register
being down is a failure only when every register is.
Google Books is asked last because it answers anonymous callers only until a
@@ -127,7 +130,8 @@ id is asked only when every form of the ISBN asked was answered with no cover; i
asking by ISBN failed, that failure is the result.
What comes back is checked before it is written: a JPEG or PNG by its bytes
as well as by what the server says, no larger than ten megabytes, reached in at
as well as by what the server says, whole to its end marker (`FF D9` for a
JPEG, `IEND` for a PNG), no larger than ten megabytes, reached in at
most three redirects, each over HTTPS to `covers.openlibrary.org`,
`archive.org` or a host under `archive.org`, and never to an address by number.
One request is made at a time, a `429` or `503` is waited out for as long as