mirror of
https://git.pumpin.online/funkpower/catena.git
synced 2026-11-13 23:55:40 +00:00
bibfetch: a cover id is the edition's own, and a fetch has a deadline
Review id 3 on catena#2. P0: Record.cover_id came from the search index's cover_i, the cover chosen for the whole work, and the index is asked first, so cover() fell back to another edition's image (Penguin War and Peace, 9780140447934, got Standard Ebooks' Maude cover 12621906). cover_i is no longer asked for or read; cover_id is the first positive id in the edition record's covers, or None. An ISBN request that fails, as opposed to a 404, now ends the fetch with that failure instead of going on to the cover id. Redirects go only over HTTPS on the standard port to covers.openlibrary.org, archive.org or a host under archive.org, and never to an address by number; the covers store the fetch began at is still allowed. The 1 KB size floor is gone. With default=false a missing cover is a 404, and the stand-in without it is a 43-byte GIF the media type check refuses, so the floor could only drop real Small covers. Covers.deadline (DEADLINE, two minutes) bounds a whole fetch: every address, attempt, redirect, body and wait. It starts once the fetch holds the process lock. The timeout test no longer times a span that includes the lock wait.
This commit is contained in:
1 parent
19e5f3ddc3
commit
1abed4632d
4 files changed
+349
-73
No files matched your search
@@ -118,14 +118,21 @@ writes the image. The thirteen-digit form is asked first and the ten-digit
|
||||
form second, because a book is often filed under only one of them; an ISBN
|
||||
whose check digit is wrong is asked about as written, since converting it
|
||||
would give another book's number. `--cover-id` names an Open Library cover id
|
||||
to ask by last, which is what the `cover_id` field of a lookup record holds.
|
||||
to ask by last, which is what the `cover_id` field of a lookup record holds:
|
||||
the first cover in Open Library's record of that edition, never the cover its
|
||||
index picked for the whole work, which is often another edition's. The cover
|
||||
id is asked only when every form of the ISBN asked was answered with no cover; if
|
||||
asking by ISBN failed, that failure is the result.
|
||||
|
||||
What comes back is checked before it is written: a JPEG or PNG by its bytes
|
||||
as well as by what the server says, between one kilobyte (Open Library's
|
||||
stand-in for a missing cover is smaller) and ten megabytes, reached in at most
|
||||
three redirects that never leave HTTPS. One request is made at a time, and a
|
||||
`429` or `503` is waited out for as long as its `Retry-After` says, up to a
|
||||
minute.
|
||||
as well as by what the server says, no larger than ten megabytes, reached in at
|
||||
most three redirects, each over HTTPS to `covers.openlibrary.org`,
|
||||
`archive.org` or a host under `archive.org`, and never to an address by number.
|
||||
One request is made at a time, a `429` or `503` is waited out for as long as
|
||||
its `Retry-After` says, up to a minute, and the whole fetch, every attempt and
|
||||
redirect included, stops after two minutes. The two minutes start once it is
|
||||
the fetch's turn, so a fetch queued behind others in the same process can wait
|
||||
that long again first.
|
||||
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
|
||||
Reference in new issue
Block a user