bibfetch: a cover id is the edition's own, and a fetch has a deadline

Review id 3 on catena#2.

P0: Record.cover_id came from the search index's cover_i, the cover
chosen for the whole work, and the index is asked first, so cover()
fell back to another edition's image (Penguin War and Peace,
9780140447934, got Standard Ebooks' Maude cover 12621906). cover_i is
no longer asked for or read; cover_id is the first positive id in the
edition record's covers, or None. An ISBN request that fails, as
opposed to a 404, now ends the fetch with that failure instead of
going on to the cover id.

Redirects go only over HTTPS on the standard port to
covers.openlibrary.org, archive.org or a host under archive.org, and
never to an address by number; the covers store the fetch began at is
still allowed.

The 1 KB size floor is gone. With default=false a missing cover is a
404, and the stand-in without it is a 43-byte GIF the media type check
refuses, so the floor could only drop real Small covers.

Covers.deadline (DEADLINE, two minutes) bounds a whole fetch: every
address, attempt, redirect, body and wait. It starts once the fetch
holds the process lock. The timeout test no longer times a span that
includes the lock wait.
This commit is contained in:
funkpower committed 2026-11-13 15:04:34 -05:00
1 parent 19e5f3ddc3
commit 1abed4632d
4 files changed
+349 -73

No files matched your search

+13 -6
View File
@@ -118,14 +118,21 @@ writes the image. The thirteen-digit form is asked first and the ten-digit
form second, because a book is often filed under only one of them; an ISBN
whose check digit is wrong is asked about as written, since converting it
would give another book's number. `--cover-id` names an Open Library cover id
to ask by last, which is what the `cover_id` field of a lookup record holds.
to ask by last, which is what the `cover_id` field of a lookup record holds:
the first cover in Open Library's record of that edition, never the cover its
index picked for the whole work, which is often another edition's. The cover
id is asked only when every form of the ISBN asked was answered with no cover; if
asking by ISBN failed, that failure is the result.
What comes back is checked before it is written: a JPEG or PNG by its bytes
as well as by what the server says, between one kilobyte (Open Library's
stand-in for a missing cover is smaller) and ten megabytes, reached in at most
three redirects that never leave HTTPS. One request is made at a time, and a
`429` or `503` is waited out for as long as its `Retry-After` says, up to a
minute.
as well as by what the server says, no larger than ten megabytes, reached in at
most three redirects, each over HTTPS to `covers.openlibrary.org`,
`archive.org` or a host under `archive.org`, and never to an address by number.
One request is made at a time, a `429` or `503` is waited out for as long as
its `Retry-After` says, up to a minute, and the whole fetch, every attempt and
redirect included, stops after two minutes. The two minutes start once it is
the fetch's turn, so a fetch queued behind others in the same process can wait
that long again first.
| Option | Effect |
|---|---|