Merge remote-tracking branch 'origin/main' into bibfetch-search

A book that arrives as a file often has no identifier to ask about, only
a title and perhaps a name. `bibfetch search --title T [--author A]
[--limit N]` asks Open Library's index which books it has under those,
and Google Books only when Open Library has nothing, and prints one
record per candidate in the order the register gave them. From Rust,
`bibfetch::search(&agent, title, author, limit, attempts)`.

Each candidate is one edition. The index answers with works and, asked
for `editions.*`, names the edition of each that best matches; the ISBN,
publisher, language, year and cover come from that edition, the title,
authors, subjects and length from the work. The work's own ISBN list
mixes editions: the first War and Peace work lists four, while the
edition matched is a 1917 printing with none.

`id` is the first ISBN whose check digit agrees, in thirteen digits,
thirteen-digit entries taken first. A candidate with none is named by
the register's key: `olid` (OL…M for the edition, OL…W for the work when
no edition was named) or `googlebooks` (a volume id). Google's `OTHER`
shelfmarks are never read as ISBNs.

`cover_id` keeps the meaning it has in a lookup record: the edition's
own first cover, taken from `editions.cover_i` and only when the record
names that edition. The work's `cover_i` and `cover_edition_key` are not
asked for. A work with no edition named, and every Google Books
candidate, has none.

Open Library having nothing stands when Google Books is down; both down
is exit 69. Exit 0 a candidate, 1 none, 65 an empty title.

The retrying GET that lookups use is now `ask::get`, shared by both, and
no longer sleeps after its last attempt.

README.md

@@ -34,7 +34,7 @@ Each tool installs by name, so you can take one without taking the rest.
 | [`vocabmerge`](crates/bibclean/README.md#vocabmerge--the-other-command) | The near-duplicate headings in a catalogue's vocabulary |
 | [`seriesof`](crates/seriesof/README.md) | The series a book belongs to, and which volume this is |
 | [`idnorm`](crates/idnorm/README.md) | DOIs, ISSNs, ISBNs and arXiv ids, read and checked |
-| [`bibfetch`](crates/bibfetch/README.md) | What a register says about an identifier |
+| [`bibfetch`](crates/bibfetch/README.md) | What a register says about an identifier, and which books it has under a title |
 | [`citefind`](crates/citefind/README.md) | The references in a bibliography or a page of notes |
 | [`mdscrub`](crates/mdscrub/README.md) | Converted markdown in, clean markdown out |
 | [`mdsplit`](crates/mdsplit/README.md) | One markdown file into the works inside it |

crates/bibfetch/README.md

@@ -20,6 +20,8 @@ So this asks, and that is the only thing it does.
 - **Filling in a reference** you have only the identifier for.
 - **Before citing something**, to check the year and the authors are what you
   think they are.
+- **Cataloguing a book that came with no identifier**, to find the editions a
+  register has under its title, and an ISBN to file it by.
 
 ## Install
 
@@ -148,6 +150,77 @@ From Rust, `bibfetch::cover(&agent, isbn, cover_id, size)` returns
 `Ok(None)` when there is no cover, and a [`Cover`](src/cover.rs) with the
 bytes, the media type and the address asked when there is.
 
+## Searching by title
+
+```
+$ bibfetch search --title 'The First World War' --author Howard --limit 2
+{"v":1,"id":"9780192853622","kind":"isbn","found":true,"source":"openlibrary","title":"The First World War","authors":["Michael Eliot Howard"],"year":2002,"container":"Oxford University Press","subjects":["World War, 1914-1918"],"language":"eng","pages":154,"cover_id":1123124}
+{"v":1,"id":"9780199205592","kind":"isbn","found":true,"source":"openlibrary","title":"The First World War: A Very Short Introduction","authors":["Michael Howard"],"year":2007,"container":"Oxford University Press","language":"eng","cover_id":14606475}
+```
+
+A book that arrives as a file often has no identifier to ask about, only a
+title and perhaps a name. This asks which books a register has under those,
+and prints one record per candidate: Open Library's index first, and Google
+Books only when Open Library has nothing.
+
+Each candidate is one edition. Open Library's index answers with works, and
+names the edition of each that best matches the search; what belongs to an
+edition — its ISBN, publisher, language, year and cover — is that edition's,
+and what belongs to the work — the title, the authors, the subjects and the
+length — is the work's. Google Books answers with volumes, which are editions
+already.
+
+| Field | When the candidate has an ISBN | When it has none |
+|---|---|---|
+| `id` | The first ISBN whose check digit agrees, in thirteen digits; one written in thirteen is taken over one written in ten | The register's own key: an Open Library id such as `OL37863121M` for the edition, or `OL267171W` for the work when no edition was named; a Google Books volume id |
+| `kind` | `isbn` | `olid` for Open Library, `googlebooks` for Google Books |
+
+An ISBN whose check digit is wrong is passed over rather than corrected, and
+Google's shelfmarks, typed `OTHER`, are not read as ISBNs even when their
+digits would pass for one. A record of kind `isbn` can be handed straight back
+to `bibfetch` for everything the registers say about that edition; `olid` and
+`googlebooks` are names no lookup here answers about.
+
+`cover_id` is the same thing it is in a lookup record: the first cover in
+Open Library's record of the edition the record names, by that edition's ISBN
+or its key, so `bibfetch cover --cover-id` fetches that edition's cover and no
+other. The cover the index picked for the whole work is never asked for,
+because it is often another edition's. A work the index names no edition of
+gives the first ISBN it lists that reads, which may be any edition's, and no
+cover id; Google Books knows nothing of Open Library's covers and gives none.
+
+The candidates are what the register said, in the order it said them. This
+makes no judgement about which one is the book in hand, and keeps one that is
+plainly another book, because a title is not an identifier: *The Hamlet* comes
+back for *Hamlet*. Deciding whether a candidate is the same work is a
+comparison of titles, and [`bibclean`](../bibclean/README.md)'s `same_work`
+is one.
+
+```sh
+bibfetch search --title 'Anathem' --author Stephenson \
+  | jq -r 'select(.kind == "isbn") | .id' | head -1
+```
+
+Open Library answering that it has nothing is an answer, and it stands when
+Google Books cannot be reached; a register being down is a failure only when
+both are. Each is asked the same way a lookup asks, retried with the same
+waits. Open Library is sent the title and the author as written; Google Books
+is sent each word as its own `intitle:` or `inauthor:` term, so nothing in a
+title is read as search syntax.
+
+| Option | Effect |
+|---|---|
+| `--title <TITLE>` | The title, or as much of it as you have |
+| `--author <AUTHOR>` | An author's name, or part of one |
+| `--limit <COUNT>` | The most candidates to print, `1` to `40`. Default `10` |
+
+Exit `0` at least one candidate was printed, `1` no register has the title,
+`65` the title was empty, `69` no register could be reached, `74` the output
+could not be written.
+
+From Rust, `bibfetch::search(&agent, title, author, limit, attempts)` returns
+the candidates, empty when no register has the title.
+
 ## Reaching the network
 
 This and [`emb`](../emb/README.md) are the only two tools here that do. Both
@@ -156,10 +229,11 @@ something that happens on the way to something else — which is the condition
 under which [the contract](../../docs/contract.md) allows it at all.
 
 Nothing is cached, nothing is stored but a cover you asked to have written,
-and nothing is sent but the identifier and the address you pass to
-`--polite`.
+and nothing is sent but the identifier, or the title and author searched for,
+and the address you pass to `--polite`.
 
 ## See also
 
 - [`idnorm`](../idnorm/README.md) — reading the identifier in the first place.
-- [`bibclean`](../bibclean/README.md) — the rest of a catalogue record.
+- [`bibclean`](../bibclean/README.md) — the rest of a catalogue record, and
+  whether a candidate is the same work.

crates/bibfetch/src/ask.rs

@@ -89,10 +89,6 @@ pub fn lookup(
 }
 
 /// Ask a register, retrying a failure that might not be one.
-///
-/// Waits a second before the second attempt and two before the third: long
-/// enough for a register that is rate-limiting, short enough that one which
-/// is simply down does not hold up a pipeline.
 fn ask(
     agent: &ureq::Agent,
     source: Source,
@@ -100,39 +96,48 @@ fn ask(
     kind: &str,
     attempts: u32,
 ) -> Result<crate::Record, Error> {
-    let mut last = Error::Unreachable("no attempt was made".into());
+    match get(agent, &source.url(id), attempts)? {
+        Some(body) => source.read(id, kind, &body),
+        // A register that does not have it says so with a status, and that is
+        // an answer rather than a failure.
+        None => Ok(crate::missing(id, kind, source)),
+    }
+}
 
-    for attempt in 0..attempts.max(1) {
-        match fetch(agent, source, id, kind) {
-            Ok(record) => return Ok(record),
-            // A shape this cannot read will not read any better next time.
-            Err(e @ Error::Shape(_)) => return Err(e),
+/// The body at an address, or `None` when the register says it has no such
+/// thing; a failure is asked again.
+///
+/// Waits a second before the second attempt and two before the third: long
+/// enough for a register that is rate-limiting, short enough that one which
+/// is simply down does not hold up a pipeline. Reading what came back is left
+/// to the caller, because an answer this cannot read will not read any better
+/// next time.
+pub(crate) fn get(agent: &ureq::Agent, url: &str, attempts: u32) -> Result<Option<String>, Error> {
+    let attempts = attempts.max(1);
+    let mut last = Error::Unreachable("no attempt was made".into());
+    for attempt in 0..attempts {
+        match once(agent, url) {
+            Ok(answer) => return Ok(answer),
             Err(e) => last = e,
         }
-        std::thread::sleep(Duration::from_secs(1 << attempt));
+        if attempt + 1 < attempts {
+            std::thread::sleep(Duration::from_secs(1 << attempt.min(5)));
+        }
     }
     Err(last)
 }
 
-fn fetch(
-    agent: &ureq::Agent,
-    source: Source,
-    id: &str,
-    kind: &str,
-) -> Result<crate::Record, Error> {
-    let response = agent.get(source.url(id)).call();
-    let mut response = match response {
+fn once(agent: &ureq::Agent, url: &str) -> Result<Option<String>, Error> {
+    let mut response = match agent.get(url).call() {
         Ok(response) => response,
-        // A register that does not have it says so with a status, and that is
-        // an answer rather than a failure.
-        Err(ureq::Error::StatusCode(404)) => return Ok(crate::missing(id, kind, source)),
+        Err(ureq::Error::StatusCode(404)) => return Ok(None),
         Err(e) => return Err(Error::Unreachable(e.to_string())),
     };
-    let body = response
+    response
         .body_mut()
         .read_to_string()
-        .map_err(|e| Error::Unreachable(e.to_string()))?;
-    source.read(id, kind, &body)
+        .map(Some)
+        .map_err(|e| Error::Unreachable(e.to_string()))
 }
 
 /// Who is asking.

crates/bibfetch/src/lib.rs

@@ -12,9 +12,14 @@
 //! them: Crossref for a DOI, the arXiv for a preprint, OpenLibrary for a
 //! book. There is no register for an ISSN that answers without a key, so an
 //! ISSN is looked up as the journal of a work rather than on its own.
+//!
+//! A book with no identifier at all can still be looked for by its title,
+//! with [`search()`]: Open Library's index, then Google Books, each candidate
+//! a record of one edition, in the order the register gave them.
 
 pub mod ask;
 pub mod cover;
+pub mod search;
 pub mod source;
 
 use std::time::Duration;
@@ -23,6 +28,7 @@ use schemars::JsonSchema;
 use serde::{Deserialize, Serialize};
 
 pub use cover::{Cover, CoverSize, cover};
+pub use search::search;
 pub use source::Source;
 
 /// How long to wait on a register before giving up.

crates/bibfetch/src/main.rs

@@ -30,7 +30,8 @@ Example:
 
     idnorm refs.txt | bibfetch | jq -r 'select(.found) | .title'
     idnorm --kind doi refs.txt | bibfetch --polite [email protected]
-    bibfetch cover 978-0-306-40615-7 -o cover.jpg"
+    bibfetch cover 978-0-306-40615-7 -o cover.jpg
+    bibfetch search --title 'The First World War' --author Howard"
 )]
 struct Args {
     #[command(subcommand)]
@@ -74,6 +75,26 @@ enum Command {
         #[arg(long, value_name = "ID")]
         cover_id: Option<u64>,
     },
+    /// Find the books a register has under a title, and an author.
+    ///
+    /// Prints one record per candidate edition, in the order the register
+    /// gave them: Open Library's index, or Google Books when that has
+    /// nothing. Which one is the book in hand is left to you.
+    ///
+    /// Exit codes: 0 at least one candidate, 1 none, 65 no title,
+    /// 69 no register could be reached, 74 the output could not be written.
+    Search {
+        /// The title, or as much of it as you have.
+        #[arg(long)]
+        title: String,
+        /// An author's name, or part of one.
+        #[arg(long)]
+        author: Option<String>,
+        /// The most candidates to print.
+        #[arg(long, default_value_t = 10, value_name = "COUNT",
+              value_parser = clap::value_parser!(u16).range(1..=bibfetch::search::MAX_LIMIT as i64))]
+        limit: u16,
+    },
 }
 
 #[derive(Clone, Copy, clap::ValueEnum)]
@@ -96,14 +117,27 @@ fn main() -> ExitCode {
     }
     .agent();
 
-    if let Some(Command::Cover {
-        isbn,
-        output,
-        size,
-        cover_id,
-    }) = &args.command
-    {
-        return cover(&agent, args.attempts, isbn, output, *size, *cover_id);
+    match &args.command {
+        Some(Command::Cover {
+            isbn,
+            output,
+            size,
+            cover_id,
+        }) => return cover(&agent, args.attempts, isbn, output, *size, *cover_id),
+        Some(Command::Search {
+            title,
+            author,
+            limit,
+        }) => {
+            return search(
+                &agent,
+                args.attempts,
+                title,
+                author.as_deref(),
+                usize::from(*limit),
+            );
+        }
+        None => {}
     }
 
     let text = match catena_cli::read_text(args.path.as_deref()) {
@@ -197,6 +231,36 @@ fn cover(
     }
 }
 
+/// Search by title and print the candidates.
+fn search(
+    agent: &ureq::Agent,
+    attempts: u32,
+    title: &str,
+    author: Option<&str>,
+    limit: usize,
+) -> ExitCode {
+    match bibfetch::search(agent, title, author, limit, attempts) {
+        Ok(found) => {
+            let wrote = catena_cli::stream("bibfetch", &found);
+            if format!("{wrote:?}") != format!("{:?}", ExitCode::from(exit::OK)) {
+                return wrote;
+            }
+            if found.is_empty() {
+                eprintln!("bibfetch: no register has {title:?}");
+                return ExitCode::from(exit::NO);
+            }
+            ExitCode::from(exit::OK)
+        }
+        Err(why) => {
+            eprintln!("bibfetch: {why}");
+            ExitCode::from(match why {
+                bibfetch::Error::Invalid(_) => exit::DATA,
+                _ => exit::UNAVAILABLE,
+            })
+        }
+    }
+}
+
 /// The identifier and its kind out of one record.
 fn read_identifier(line: &str) -> Result<(String, String), String> {
     let value: serde_json::Value =

crates/bibfetch/src/search.rs

@@ -0,0 +1,780 @@
+//! Books by their title, for when there is no identifier to ask about.
+//!
+//! A book that arrives as a file often carries no ISBN at all, only a title
+//! and perhaps a name. A register can still be asked which books it has under
+//! those, and what it answers is a list of candidates: each an edition the
+//! register holds, with whatever identifier, publisher and cover it gives.
+//!
+//! Which candidate is the book in hand is not decided here. The candidates
+//! are what the register said, in the order it said them; deciding whether
+//! one is the same work is a comparison of titles, and that belongs to
+//! whoever holds the book.
+
+use serde::Deserialize;
+
+use crate::source::{self, GoogleIdentifier, GoogleReply, SUBJECT_LIMIT, Source};
+use crate::{Error, Record};
+
+/// The most candidates one search asks for. Google Books answers with no more
+/// than forty at a time, and a title that needs more than forty candidates to
+/// find the right one needs a better title.
+pub const MAX_LIMIT: usize = 40;
+
+/// What Open Library's index is asked to say about each work, and about the
+/// one edition of it that best matches the search.
+///
+/// Not the work's own `cover_i` or `cover_edition_key`: the cover the index
+/// chose for a work is often another edition's. The edition's `cover_i` is
+/// the first cover in that edition's record, which is what a lookup by ISBN
+/// gives as `cover_id`.
+const OPEN_LIBRARY_FIELDS: &str = "key,title,subtitle,author_name,first_publish_year,publisher,\
+isbn,language,subject,number_of_pages_median,editions,editions.key,editions.isbn,\
+editions.publisher,editions.language,editions.publish_date,editions.cover_i";
+
+/// Ask which books a register has under a title, and by an author if one is
+/// named.
+///
+/// Open Library's index first, and Google Books only when that answered with
+/// nothing. A register being down is a failure only when every register
+/// asked is: Open Library answering that it has nothing is an answer, and it
+/// stands when Google Books cannot be reached. Asks nothing when `limit` is
+/// zero; a `limit` over [`MAX_LIMIT`] is taken as that.
+pub fn search(
+    agent: &ureq::Agent,
+    title: &str,
+    author: Option<&str>,
+    limit: usize,
+    attempts: u32,
+) -> Result<Vec<Record>, Error> {
+    search_with(title, author, limit, |url| {
+        crate::ask::get(agent, url, attempts)
+    })
+}
+
+/// [`search`], with how to fetch an address passed in.
+fn search_with(
+    title: &str,
+    author: Option<&str>,
+    limit: usize,
+    mut get: impl FnMut(&str) -> Result<Option<String>, Error>,
+) -> Result<Vec<Record>, Error> {
+    let title = title.trim();
+    if title.is_empty() {
+        return Err(Error::Invalid("no title to search for".into()));
+    }
+    let author = author.map(str::trim).filter(|author| !author.is_empty());
+    let limit = limit.min(MAX_LIMIT);
+    if limit == 0 {
+        return Ok(Vec::new());
+    }
+
+    let mut fetch = |url: String| match get(&url)? {
+        Some(body) => Ok(body),
+        // A search that matches nothing answers with an empty list. A 404 is
+        // the address being wrong, not the book being absent.
+        None => Err(Error::Unreachable(format!("{url}: answered 404"))),
+    };
+
+    let open_library =
+        fetch(open_library_url(title, author, limit)).and_then(|body| open_library(&body, limit));
+    if open_library.as_ref().is_ok_and(|found| !found.is_empty()) {
+        return open_library;
+    }
+
+    let google = match google_url(title, author, limit) {
+        Some(url) => fetch(url).and_then(|body| google_books(&body, limit)),
+        // Nothing in the title Google could be asked by.
+        None => Ok(Vec::new()),
+    };
+    match (open_library, google) {
+        (_, Ok(found)) if !found.is_empty() => Ok(found),
+        (Ok(none), _) => Ok(none),
+        (Err(why), _) => Err(why),
+    }
+}
+
+/// Where to ask Open Library.
+fn open_library_url(title: &str, author: Option<&str>, limit: usize) -> String {
+    let author = author
+        .map(|author| format!("&author={}", source::encode(author)))
+        .unwrap_or_default();
+    format!(
+        "https://openlibrary.org/search.json?title={}{author}&fields={OPEN_LIBRARY_FIELDS}&limit={limit}",
+        source::encode(title),
+    )
+}
+
+/// Where to ask Google Books, or `None` when there is nothing to ask by.
+///
+/// Every word is its own `intitle:` or `inauthor:` term, so no character of
+/// the title can open a phrase, negate a term or name another field: a word is
+/// the letters, digits and apostrophes in it, and anything else separates
+/// words.
+fn google_url(title: &str, author: Option<&str>, limit: usize) -> Option<String> {
+    let terms: Vec<String> = words(title)
+        .map(|word| format!("intitle:{word}"))
+        .chain(
+            author
+                .into_iter()
+                .flat_map(words)
+                .map(|word| format!("inauthor:{word}")),
+        )
+        .collect();
+    if !terms.iter().any(|term| term.starts_with("intitle:")) {
+        return None;
+    }
+    Some(format!(
+        "https://www.googleapis.com/books/v1/volumes?q={}&maxResults={limit}",
+        source::encode(&terms.join(" "))
+    ))
+}
+
+fn words(text: &str) -> impl Iterator<Item = &str> {
+    text.split(|c: char| !(c.is_alphanumeric() || c == '\''))
+        .map(|word| word.trim_matches('\''))
+        .filter(|word| !word.is_empty())
+}
+
+/// The first ISBN in a list whose check digit agrees, in its thirteen-digit
+/// form. One written in thirteen digits is taken over one written in ten.
+fn first_isbn<'a>(isbns: impl Iterator<Item = &'a str> + Clone) -> Option<String> {
+    isbns
+        .clone()
+        .filter(|isbn| idnorm::isbn::digits(isbn).len() == 13)
+        .find_map(idnorm::isbn::to_13)
+        .or_else(|| isbns.into_iter().find_map(idnorm::isbn::to_13))
+}
+
+#[derive(Deserialize)]
+struct OpenLibraryReply {
+    #[serde(default)]
+    docs: Vec<Work>,
+}
+
+#[derive(Deserialize)]
+struct Work {
+    #[serde(default)]
+    key: Option<String>,
+    #[serde(default)]
+    title: Option<String>,
+    #[serde(default)]
+    subtitle: Option<String>,
+    #[serde(default)]
+    author_name: Vec<String>,
+    #[serde(default)]
+    first_publish_year: Option<i64>,
+    #[serde(default)]
+    publisher: Vec<String>,
+    #[serde(default)]
+    isbn: Vec<String>,
+    #[serde(default)]
+    language: Vec<String>,
+    #[serde(default)]
+    subject: Vec<String>,
+    #[serde(default)]
+    number_of_pages_median: Option<u32>,
+    #[serde(default)]
+    editions: Option<Editions>,
+}
+
+#[derive(Deserialize)]
+struct Editions {
+    #[serde(default)]
+    docs: Vec<Edition>,
+}
+
+#[derive(Deserialize)]
+struct Edition {
+    #[serde(default)]
+    key: Option<String>,
+    #[serde(default)]
+    isbn: Vec<String>,
+    #[serde(default)]
+    publisher: Vec<String>,
+    #[serde(default)]
+    language: Vec<String>,
+    #[serde(default)]
+    publish_date: Vec<String>,
+    #[serde(default)]
+    cover_i: Option<i64>,
+}
+
+/// The last segment of an Open Library key: `OL3585423M` out of
+/// `/books/OL3585423M`. The letter at the end says what it names, `M` an
+/// edition and `W` a work.
+fn olid(key: &str) -> Option<String> {
+    key.rsplit('/')
+        .next()
+        .filter(|id| !id.is_empty())
+        .map(str::to_string)
+}
+
+/// Read Open Library's answer into one candidate per work, in its order.
+///
+/// What belongs to an edition — its ISBN, publisher, language, year and
+/// cover — is taken from the edition the index chose as matching best; what
+/// belongs to the work — its title, authors, subjects and length — from the
+/// work. A work the index names no edition of gives its first ISBN that reads,
+/// which may be any edition's, and no cover id.
+fn open_library(body: &str, limit: usize) -> Result<Vec<Record>, Error> {
+    let reply: OpenLibraryReply =
+        serde_json::from_str(body).map_err(|e| Error::Shape(e.to_string()))?;
+    Ok(reply
+        .docs
+        .into_iter()
+        .filter_map(candidate)
+        .take(limit)
+        .collect())
+}
+
+fn candidate(work: Work) -> Option<Record> {
+    let edition = work.editions.and_then(|e| e.docs.into_iter().next());
+
+    // The cover is kept only when the record names the edition it is the
+    // cover of, by that edition's ISBN or its key.
+    let (id, kind, edition_named) = match &edition {
+        Some(edition) => match first_isbn(edition.isbn.iter().map(String::as_str)) {
+            Some(isbn) => (isbn, "isbn", true),
+            None => match edition.key.as_deref().and_then(olid) {
+                Some(key) => (key, "olid", true),
+                None => (work.key.as_deref().and_then(olid)?, "olid", false),
+            },
+        },
+        None => match first_isbn(work.isbn.iter().map(String::as_str)) {
+            Some(isbn) => (isbn, "isbn", false),
+            // A work with no key cannot be named, so it is not a candidate.
+            None => (work.key.as_deref().and_then(olid)?, "olid", false),
+        },
+    };
+
+    let mut record = source::blank(&id, kind, Source::OpenLibrarySearch);
+    record.title = match (work.title, work.subtitle) {
+        (Some(title), Some(subtitle)) => Some(format!("{title}: {subtitle}")),
+        (title, _) => title,
+    };
+    record.authors = work.author_name;
+    record.subjects = work.subject;
+    record.subjects.truncate(SUBJECT_LIMIT);
+    record.pages = work.number_of_pages_median;
+
+    let (publisher, language, year) = match edition {
+        Some(edition) => {
+            if edition_named {
+                record.cover_id = edition
+                    .cover_i
+                    .and_then(|id| u64::try_from(id).ok())
+                    .filter(|id| *id > 0);
+            }
+            (
+                edition.publisher.into_iter().next(),
+                edition.language.into_iter().next(),
+                edition
+                    .publish_date
+                    .iter()
+                    .find_map(|date| source::year_in(date)),
+            )
+        }
+        None => (None, None, None),
+    };
+    record.container = publisher.or_else(|| work.publisher.into_iter().next());
+    record.language = language.or_else(|| work.language.into_iter().next());
+    record.year = year.or(work.first_publish_year);
+    Some(record)
+}
+
+/// Read Google Books' answer into one candidate per volume, in its order.
+fn google_books(body: &str, limit: usize) -> Result<Vec<Record>, Error> {
+    let reply: GoogleReply = serde_json::from_str(body).map_err(|e| Error::Shape(e.to_string()))?;
+    Ok(reply
+        .items
+        .into_iter()
+        .filter_map(|item| {
+            // Only what Google types as an ISBN: its `OTHER` identifiers are
+            // library shelfmarks, some of which are ten digits long.
+            let isbns = item
+                .volume_info
+                .identifiers
+                .iter()
+                .filter(|GoogleIdentifier { kind, .. }| kind == "ISBN_13" || kind == "ISBN_10")
+                .map(|identifier| identifier.identifier.as_str());
+            let (id, kind) = match first_isbn(isbns) {
+                Some(isbn) => (isbn, "isbn"),
+                None => (item.id.filter(|id| !id.is_empty())?, "googlebooks"),
+            };
+            Some(source::google_volume(&id, kind, item.volume_info))
+        })
+        .take(limit)
+        .collect())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// Open Library's answer to `title=the first world war&author=howard`,
+    /// recorded 2026-11-13 with the fields above and `limit=2`, the ISBN and
+    /// subject lists shortened.
+    const HOWARD: &str = r#"{"numFound": 9, "start": 0, "numFoundExact": true, "num_found": 9,
+        "documentation_url": "https://openlibrary.org/dev/docs/api/search", "q": "", "offset": null,
+        "docs": [
+          {"author_name": ["Michael Eliot Howard"], "first_publish_year": 2002,
+           "isbn": ["9780192853622", "0192853627"], "key": "/works/OL1382726W",
+           "language": ["eng"], "number_of_pages_median": 154,
+           "publisher": ["Oxford University Press"], "title": "The First World War",
+           "subject": ["World War, 1914-1918"],
+           "editions": {"numFound": 1, "start": 0, "numFoundExact": true, "docs": [
+             {"key": "/books/OL3585423M", "cover_i": 1123124, "language": ["eng"],
+              "publisher": ["Oxford University Press"], "publish_date": ["2002"],
+              "isbn": ["0192853627", "9780192853622"]}]}},
+          {"author_name": ["Michael Howard"], "first_publish_year": 2007,
+           "isbn": ["0199205590", "9780199205592"], "key": "/works/OL4282399W",
+           "language": ["eng"], "publisher": ["Oxford University Press"],
+           "subtitle": "A Very Short Introduction", "title": "The First World War",
+           "editions": {"numFound": 1, "start": 0, "numFoundExact": true, "docs": [
+             {"key": "/books/OL7403004M", "cover_i": 14606475, "language": ["eng"],
+              "publisher": ["Oxford University Press"], "publish_date": ["20 May 2007"],
+              "isbn": ["9780199205592", "0199205590"]}]},
+           "subject": []}]}"#;
+
+    /// Open Library's answer to `title=war and peace&author=tolstoy`, recorded
+    /// 2026-11-13, first work only. The edition the index matched best is a
+    /// 1917 printing that has no ISBN, though other editions of the work do.
+    const TOLSTOY: &str = r#"{"numFound": 146, "docs": [
+          {"author_name": ["Leo Tolstoy"], "first_publish_year": 1867,
+           "isbn": ["9781524881993", "1645178889"], "key": "/works/OL267171W",
+           "title": "War and Peace",
+           "editions": {"numFound": 487, "docs": [
+             {"key": "/books/OL37863121M", "cover_i": 12736340, "language": ["eng"],
+              "publisher": ["Thomas Y. Crowell"], "publish_date": ["1917"]}]}}]}"#;
+
+    /// Open Library's answer when nothing has the title.
+    const NOTHING: &str = r#"{"numFound": 0, "start": 0, "numFoundExact": true, "num_found": 0,
+        "documentation_url": "https://openlibrary.org/dev/docs/api/search", "q": "",
+        "offset": null, "docs": []}"#;
+
+    /// Google Books' answer for a volume, in the shape its API documents.
+    const GOOGLE: &str = r#"{"kind": "books#volumes", "totalItems": 2, "items": [
+          {"kind": "books#volume", "id": "zyTCAlFPjgYC", "volumeInfo": {
+             "title": "The Google Story", "authors": ["David A. Vise", "Mark Malseed"],
+             "publisher": "Random House Digital, Inc.", "publishedDate": "2005-11-15",
+             "industryIdentifiers": [
+               {"type": "ISBN_10", "identifier": "055380457X"},
+               {"type": "ISBN_13", "identifier": "9780553804577"}],
+             "pageCount": 207, "categories": ["Browsers (Computer programs)"],
+             "language": "en"}},
+          {"kind": "books#volume", "id": "a1b2C3d4", "volumeInfo": {
+             "title": "The Google Story", "subtitle": "A Dissertation",
+             "publishedDate": "2009",
+             "industryIdentifiers": [
+               {"type": "OTHER", "identifier": "UOM:0306406152"}]}}]}"#;
+
+    /// Fetch from a table of recorded bodies, keeping every address asked.
+    fn recorded<'a>(
+        asked: &'a mut Vec<String>,
+        bodies: &'a [(&'a str, Result<&'a str, Error>)],
+    ) -> impl FnMut(&str) -> Result<Option<String>, Error> + 'a {
+        move |url| {
+            asked.push(url.to_string());
+            let (_, body) = bodies
+                .iter()
+                .find(|(host, _)| url.contains(host))
+                .unwrap_or_else(|| panic!("asked an address nothing was recorded for: {url}"));
+            body.clone().map(|body| Some(body.to_string()))
+        }
+    }
+
+    fn down() -> Error {
+        Error::Unreachable("connection refused".into())
+    }
+
+    #[test]
+    fn each_work_open_library_has_is_a_candidate_named_by_its_isbn() {
+        let mut asked = Vec::new();
+        let bodies = [("openlibrary.org", Ok(HOWARD))];
+        let found = search_with(
+            "The First World War",
+            Some("Howard"),
+            5,
+            recorded(&mut asked, &bodies),
+        )
+        .expect("candidates");
+        assert_eq!(
+            asked.len(),
+            1,
+            "Google is not asked when Open Library answers"
+        );
+
+        assert_eq!(found.len(), 2);
+        let first = &found[0];
+        assert!(first.found);
+        assert_eq!(first.v, corpus_core::V);
+        assert_eq!(first.source, "openlibrary");
+        // Written ten-digit first by the edition, given in thirteen.
+        assert_eq!(
+            (first.id.as_str(), first.kind.as_str()),
+            ("9780192853622", "isbn")
+        );
+        assert_eq!(first.title.as_deref(), Some("The First World War"));
+        assert_eq!(first.authors, ["Michael Eliot Howard"]);
+        assert_eq!(first.year, Some(2002));
+        assert_eq!(first.container.as_deref(), Some("Oxford University Press"));
+        assert_eq!(first.language.as_deref(), Some("eng"));
+        assert_eq!(first.subjects, ["World War, 1914-1918"]);
+        assert_eq!(first.pages, Some(154));
+        assert_eq!(first.cover_id, Some(1123124));
+
+        let second = &found[1];
+        assert_eq!(second.id, "9780199205592");
+        assert_eq!(
+            second.title.as_deref(),
+            Some("The First World War: A Very Short Introduction")
+        );
+        // The edition's year, not the work's first.
+        assert_eq!(second.year, Some(2007));
+        // The first cover in OL7403004M's own record; the work's cover is
+        // 1137779, which is not.
+        assert_eq!(second.cover_id, Some(14606475));
+    }
+
+    #[test]
+    fn the_candidates_stay_in_the_order_the_register_gave_them() {
+        let mut asked = Vec::new();
+        let bodies = [("openlibrary.org", Ok(HOWARD))];
+        let found =
+            search_with("first world war", None, 5, recorded(&mut asked, &bodies)).expect("ok");
+        let ids: Vec<&str> = found.iter().map(|r| r.id.as_str()).collect();
+        assert_eq!(ids, ["9780192853622", "9780199205592"]);
+        let found =
+            search_with("first world war", None, 1, recorded(&mut asked, &bodies)).expect("ok");
+        assert_eq!(found.len(), 1);
+        assert_eq!(found[0].id, "9780192853622");
+    }
+
+    #[test]
+    fn an_edition_with_no_isbn_is_named_by_its_key_and_keeps_its_cover() {
+        let mut asked = Vec::new();
+        let bodies = [("openlibrary.org", Ok(TOLSTOY))];
+        let found = search_with(
+            "War and Peace",
+            Some("Tolstoy"),
+            5,
+            recorded(&mut asked, &bodies),
+        )
+        .expect("candidates");
+        let only = &found[0];
+        // Not 9781524881993: that is another edition's, and the publisher,
+        // year and cover here are this one's.
+        assert_eq!(
+            (only.id.as_str(), only.kind.as_str()),
+            ("OL37863121M", "olid")
+        );
+        assert_eq!(only.container.as_deref(), Some("Thomas Y. Crowell"));
+        assert_eq!(only.year, Some(1917));
+        assert_eq!(only.cover_id, Some(12736340));
+    }
+
+    #[test]
+    fn a_work_with_no_edition_named_gives_its_isbn_but_no_cover() {
+        let body = r#"{"docs": [{"key": "/works/OL1W", "title": "A Book",
+            "isbn": ["0-306-40615-2"], "publisher": ["Plenum"], "first_publish_year": 1981}]}"#;
+        let found = open_library(body, 5).expect("candidates");
+        assert_eq!(
+            (found[0].id.as_str(), found[0].kind.as_str()),
+            ("9780306406157", "isbn")
+        );
+        assert_eq!(found[0].container.as_deref(), Some("Plenum"));
+        assert_eq!(found[0].year, Some(1981));
+        assert_eq!(found[0].cover_id, None);
+    }
+
+    #[test]
+    fn a_work_with_no_isbn_and_no_edition_is_named_by_the_works_key() {
+        let body = r#"{"docs": [{"key": "/works/OL267171W", "title": "War and Peace"}]}"#;
+        let found = open_library(body, 5).expect("candidates");
+        assert_eq!(
+            (found[0].id.as_str(), found[0].kind.as_str()),
+            ("OL267171W", "olid")
+        );
+        assert_eq!(found[0].cover_id, None);
+    }
+
+    #[test]
+    fn isbns_whose_check_digit_is_wrong_are_passed_over() {
+        // 9780306406158 and 0306406153 are 9780306406157 mistyped.
+        let body = r#"{"docs": [
+            {"key": "/works/OL1W", "title": "A", "editions": {"docs": [
+              {"key": "/books/OL1M", "isbn": ["9780306406158", "0306406153"], "cover_i": 7}]}},
+            {"key": "/works/OL2W", "title": "B", "editions": {"docs": [
+              {"key": "/books/OL2M", "isbn": ["9780306406158", "0306406152"]}]}}]}"#;
+        let found = open_library(body, 5).expect("candidates");
+        assert_eq!(
+            (found[0].id.as_str(), found[0].kind.as_str()),
+            ("OL1M", "olid")
+        );
+        assert_eq!(found[0].cover_id, Some(7));
+        // The ten-digit one reads, and is given in thirteen.
+        assert_eq!(
+            (found[1].id.as_str(), found[1].kind.as_str()),
+            ("9780306406157", "isbn")
+        );
+    }
+
+    #[test]
+    fn thirteen_digits_are_taken_over_ten() {
+        // Two different ISBNs on one edition: the thirteen-digit one wins
+        // although the ten-digit one is listed first.
+        let isbns = ["0306406152", "9780199205592"];
+        assert_eq!(
+            first_isbn(isbns.into_iter()).as_deref(),
+            Some("9780199205592")
+        );
+        assert_eq!(first_isbn(["nonsense"].into_iter()), None);
+    }
+
+    #[test]
+    fn nothing_from_open_library_is_asked_of_google_books() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Ok(NOTHING)),
+            ("googleapis.com", Ok(GOOGLE)),
+        ];
+        let found = search_with(
+            "The Google Story",
+            Some("Vise"),
+            5,
+            recorded(&mut asked, &bodies),
+        )
+        .expect("candidates");
+        assert_eq!(asked.len(), 2);
+        assert!(asked[1].starts_with("https://www.googleapis.com/books/v1/volumes?q="));
+
+        assert_eq!(found.len(), 2);
+        let first = &found[0];
+        assert_eq!(first.source, "googlebooks");
+        assert_eq!(
+            (first.id.as_str(), first.kind.as_str()),
+            ("9780553804577", "isbn")
+        );
+        assert_eq!(first.authors, ["David A. Vise", "Mark Malseed"]);
+        assert_eq!(first.year, Some(2005));
+        assert_eq!(
+            first.container.as_deref(),
+            Some("Random House Digital, Inc.")
+        );
+        assert_eq!(first.pages, Some(207));
+        // Google Books knows nothing of Open Library's covers.
+        assert_eq!(first.cover_id, None);
+
+        // A shelfmark is not an ISBN, though its digits would pass for one.
+        let second = &found[1];
+        assert_eq!(
+            (second.id.as_str(), second.kind.as_str()),
+            ("a1b2C3d4", "googlebooks")
+        );
+        assert_eq!(
+            second.title.as_deref(),
+            Some("The Google Story: A Dissertation")
+        );
+    }
+
+    #[test]
+    fn nothing_from_either_register_is_no_candidates() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Ok(NOTHING)),
+            (
+                "googleapis.com",
+                Ok(r#"{"kind": "books#volumes", "totalItems": 0}"#),
+            ),
+        ];
+        let found =
+            search_with("Nothing", None, 5, recorded(&mut asked, &bodies)).expect("an answer");
+        assert!(found.is_empty());
+    }
+
+    #[test]
+    fn open_library_having_nothing_stands_when_google_books_is_down() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Ok(NOTHING)),
+            ("googleapis.com", Err(down())),
+        ];
+        let found =
+            search_with("Nothing", None, 5, recorded(&mut asked, &bodies)).expect("an answer");
+        assert!(found.is_empty());
+    }
+
+    #[test]
+    fn open_library_down_is_asked_of_google_books() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Err(down())),
+            ("googleapis.com", Ok(GOOGLE)),
+        ];
+        let found =
+            search_with("The Google Story", None, 5, recorded(&mut asked, &bodies)).expect("ok");
+        assert_eq!(found[0].source, "googlebooks");
+    }
+
+    #[test]
+    fn every_register_down_is_a_failure() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Err(down())),
+            ("googleapis.com", Err(down())),
+        ];
+        let failed = search_with("A Book", None, 5, recorded(&mut asked, &bodies));
+        assert!(matches!(failed, Err(Error::Unreachable(_))), "{failed:?}");
+    }
+
+    #[test]
+    fn an_answer_that_cannot_be_read_says_so() {
+        assert!(matches!(open_library("<html>", 5), Err(Error::Shape(_))));
+        assert!(matches!(google_books("<html>", 5), Err(Error::Shape(_))));
+    }
+
+    #[test]
+    fn the_author_is_left_out_when_there_is_none() {
+        for author in [None, Some(""), Some("   ")] {
+            let url = open_library_url(
+                "Anathem",
+                author.map(str::trim).filter(|a| !a.is_empty()),
+                5,
+            );
+            assert!(!url.contains("author="), "{url}");
+        }
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Ok(NOTHING)),
+            ("googleapis.com", Ok(GOOGLE)),
+        ];
+        search_with("Anathem", Some("  "), 5, recorded(&mut asked, &bodies)).expect("ok");
+        assert!(!asked[0].contains("author="), "{}", asked[0]);
+        assert!(!asked[1].contains("inauthor"), "{}", asked[1]);
+        assert!(asked[1].contains("q=intitle%3AAnathem&"), "{}", asked[1]);
+    }
+
+    #[test]
+    fn the_author_narrows_the_search_when_there_is_one() {
+        let url = open_library_url("Anathem", Some("Neal Stephenson"), 5);
+        assert!(
+            url.contains("?title=Anathem&author=Neal%20Stephenson&fields="),
+            "{url}"
+        );
+        let url = google_url("Anathem", Some("Neal Stephenson"), 5).expect("a url");
+        assert!(
+            url.contains("q=intitle%3AAnathem%20inauthor%3ANeal%20inauthor%3AStephenson&"),
+            "{url}"
+        );
+    }
+
+    #[test]
+    fn a_title_cannot_change_the_shape_of_the_address() {
+        let hostile = r#"Fish & Chips: "A History" #2?=x/y+z"#;
+        for url in [
+            open_library_url(hostile, Some("O'Brien & Sons"), 5),
+            google_url(hostile, Some("O'Brien & Sons"), 5).expect("a url"),
+        ] {
+            assert_eq!(url.matches('?').count(), 1, "{url}");
+            assert_eq!(url.matches('#').count(), 0, "{url}");
+            assert_eq!(url.matches('"').count(), 0, "{url}");
+            assert_eq!(url.matches('+').count(), 0, "{url}");
+            let query = url.split_once('?').expect("a query").1;
+            let names: Vec<&str> = query
+                .split('&')
+                .map(|pair| pair.split_once('=').expect("name=value").0)
+                .collect();
+            assert!(
+                names == ["title", "author", "fields", "limit"] || names == ["q", "maxResults"],
+                "{url}"
+            );
+        }
+        // Open Library is asked the title as written.
+        let url = open_library_url(hostile, None, 5);
+        assert!(
+            url.contains("title=Fish%20%26%20Chips%3A%20%22A%20History%22%20%232%3F%3Dx%2Fy%2Bz&"),
+            "{url}"
+        );
+        // Google is asked word by word, so nothing in a title is syntax.
+        let url = google_url(hostile, Some("O'Brien & Sons"), 5).expect("a url");
+        let q = "intitle:Fish intitle:Chips intitle:A intitle:History intitle:2 intitle:x \
+intitle:y intitle:z inauthor:O'Brien inauthor:Sons";
+        assert!(url.contains(&format!("q={}&", source::encode(q))), "{url}");
+    }
+
+    #[test]
+    fn a_title_outside_ascii_is_sent_as_utf8() {
+        let url = open_library_url("Преступление и наказание", Some("Dostoevský"), 5);
+        assert!(
+            url.contains("title=%D0%9F%D1%80%D0%B5%D1%81%D1%82%D1%83%D0%BF"),
+            "{url}"
+        );
+        assert!(url.contains("author=Dostoevsk%C3%BD&"), "{url}");
+        let url = google_url("Преступление и наказание", None, 5).expect("a url");
+        // Letters in any script are words.
+        assert!(url.contains("intitle%3A%D0%B8%20"), "{url}");
+    }
+
+    #[test]
+    fn a_title_with_no_words_is_not_asked_of_google_books() {
+        assert_eq!(google_url("?!", Some("Someone"), 5), None);
+        let mut asked = Vec::new();
+        let bodies = [("openlibrary.org", Ok(NOTHING))];
+        let found = search_with("?!", Some("Someone"), 5, recorded(&mut asked, &bodies))
+            .expect("an answer");
+        assert!(found.is_empty());
+        assert_eq!(asked.len(), 1);
+    }
+
+    #[test]
+    fn no_title_is_refused_without_asking() {
+        let mut asked = Vec::new();
+        let failed = search_with("   ", Some("Someone"), 5, recorded(&mut asked, &[]));
+        assert!(matches!(failed, Err(Error::Invalid(_))), "{failed:?}");
+        assert!(asked.is_empty());
+    }
+
+    #[test]
+    fn the_limit_is_asked_for_and_held_to() {
+        let mut asked = Vec::new();
+        let bodies = [
+            ("openlibrary.org", Ok(NOTHING)),
+            ("googleapis.com", Ok(GOOGLE)),
+        ];
+        let found =
+            search_with("The Google Story", None, 1, recorded(&mut asked, &bodies)).expect("ok");
+        assert_eq!(found.len(), 1);
+        assert!(asked[0].ends_with("&limit=1"), "{}", asked[0]);
+        assert!(asked[1].ends_with("&maxResults=1"), "{}", asked[1]);
+
+        let mut asked = Vec::new();
+        search_with("The Google Story", None, 500, recorded(&mut asked, &bodies)).expect("ok");
+        assert!(asked[0].ends_with("&limit=40"), "{}", asked[0]);
+        assert!(asked[1].ends_with("&maxResults=40"), "{}", asked[1]);
+
+        let mut asked = Vec::new();
+        let found =
+            search_with("The Google Story", None, 0, recorded(&mut asked, &bodies)).expect("ok");
+        assert!(found.is_empty());
+        assert!(asked.is_empty());
+    }
+
+    #[test]
+    fn the_index_is_not_asked_for_the_works_cover() {
+        let url = open_library_url("A", None, 5);
+        let fields = url
+            .split('&')
+            .find_map(|pair| pair.strip_prefix("fields="))
+            .expect("fields");
+        let fields: Vec<&str> = fields.split(',').collect();
+        assert!(fields.contains(&"editions.cover_i"));
+        assert!(!fields.contains(&"cover_i"));
+        assert!(!fields.contains(&"cover_edition_key"));
+    }
+
+    #[test]
+    fn a_404_from_a_search_is_a_failure_and_not_an_absence() {
+        let failed = search_with("A Book", None, 5, |_| Ok(None));
+        assert!(matches!(failed, Err(Error::Unreachable(_))), "{failed:?}");
+    }
+}

crates/bibfetch/src/source.rs

@@ -117,7 +117,7 @@ number_of_pages_median&limit=1",
 ///
 /// A DOI suffix may hold a slash, a question mark or a hash, and every one of
 /// those means something else in a URL.
-fn encode(text: &str) -> String {
+pub(crate) fn encode(text: &str) -> String {
     let mut out = String::with_capacity(text.len());
     for byte in text.bytes() {
         match byte {
@@ -131,7 +131,7 @@ fn encode(text: &str) -> String {
 }
 
 /// A record with the fields nothing found fills in.
-fn blank(id: &str, kind: &str, source: Source) -> Record {
+pub(crate) fn blank(id: &str, kind: &str, source: Source) -> Record {
     let mut record = crate::missing(id, kind, source);
     record.found = true;
     record
@@ -320,7 +320,7 @@ struct SearchDoc {
 /// The index lists every heading every edition has ever been given, which on a
 /// well-known book is hundreds and on Hamlet is thousands. A shelf is filed
 /// from a handful, and the first are the broadest.
-const SUBJECT_LIMIT: usize = 12;
+pub(crate) const SUBJECT_LIMIT: usize = 12;
 
 /// A search that matched nothing answers with no documents rather than a
 /// status, so an empty reply is the absence and not a failure.
@@ -348,19 +348,22 @@ fn open_library_search(id: &str, kind: &str, body: &str) -> Result<Record, Error
 }
 
 #[derive(Deserialize)]
-struct GoogleReply {
+pub(crate) struct GoogleReply {
     #[serde(default)]
-    items: Vec<GoogleItem>,
+    pub(crate) items: Vec<GoogleItem>,
 }
 
 #[derive(Deserialize)]
-struct GoogleItem {
+pub(crate) struct GoogleItem {
+    /// Google's own key for the volume.
+    #[serde(default)]
+    pub(crate) id: Option<String>,
     #[serde(rename = "volumeInfo", default)]
-    volume_info: GoogleVolume,
+    pub(crate) volume_info: GoogleVolume,
 }
 
 #[derive(Deserialize, Default)]
-struct GoogleVolume {
+pub(crate) struct GoogleVolume {
     #[serde(default)]
     title: Option<String>,
     #[serde(default)]
@@ -379,6 +382,18 @@ struct GoogleVolume {
     language: Option<String>,
     #[serde(rename = "pageCount", default)]
     page_count: Option<u32>,
+    /// `[{"type": "ISBN_13", "identifier": "…"}]`, alongside library
+    /// shelfmarks typed `OTHER`.
+    #[serde(rename = "industryIdentifiers", default)]
+    pub(crate) identifiers: Vec<GoogleIdentifier>,
+}
+
+#[derive(Deserialize)]
+pub(crate) struct GoogleIdentifier {
+    #[serde(rename = "type", default)]
+    pub(crate) kind: String,
+    #[serde(default)]
+    pub(crate) identifier: String,
 }
 
 /// A search that matched nothing answers with no items rather than a status,
@@ -388,8 +403,11 @@ fn google_books(id: &str, kind: &str, body: &str) -> Result<Record, Error> {
     let Some(item) = reply.items.into_iter().next() else {
         return Ok(crate::missing(id, kind, Source::GoogleBooks));
     };
-    let volume = item.volume_info;
+    Ok(google_volume(id, kind, item.volume_info))
+}
 
+/// One Google Books volume as a record.
+pub(crate) fn google_volume(id: &str, kind: &str, volume: GoogleVolume) -> Record {
     let mut record = blank(id, kind, Source::GoogleBooks);
     record.title = match (volume.title, volume.subtitle) {
         (Some(title), Some(subtitle)) => Some(format!("{title}: {subtitle}")),
@@ -402,11 +420,11 @@ fn google_books(id: &str, kind: &str, body: &str) -> Result<Record, Error> {
     record.subjects = volume.categories;
     record.language = volume.language;
     record.pages = volume.page_count;
-    Ok(record)
+    record
 }
 
 /// The first four-digit year in a string.
-fn year_in(text: &str) -> Option<i64> {
+pub(crate) fn year_in(text: &str) -> Option<i64> {
     let bytes: Vec<char> = text.chars().collect();
     for window in bytes.windows(4) {
         if window.iter().all(char::is_ascii_digit) {

crates/bookmeta/README.md

@@ -112,35 +112,10 @@ an EXTH block that breaks off keeps the records before the break.
 
 ## Covers
 
-`Metadata::cover` is the cover image the file carries, with its media type
-read off its first bytes: JPEG, PNG, GIF, WebP or BMP. It is for library
-callers and is not part of the JSON record, which is a line of text.
-
-A Kindle book's cover is EXTH 201, or the thumbnail, 202, when no cover is
-named, and with neither the first image record.
-
-An EPUB's is the first of these that reaches an image:
-
-1. The manifest item whose `properties` include `cover-image` (EPUB 3).
-2. The item `<meta name="cover" content="…">` names (EPUB 2). Some
-   converters write the image's address there rather than its id, and that is
-   followed too.
-3. The guide's `<reference type="cover">`, a page, and the first `<img src>`
-   or SVG `<image href>` on it. A guide that points straight at an image is
-   taken at its word.
-4. Any image in the manifest whose id or address says `cover`.
-
-A route that names something unusable — not in the archive, not an image —
-leaves a note, and the next route is asked. Addresses are resolved against the
-document they are written in, with their fragment off and their
-percent-escapes undone, and one that climbs out of the archive or names a
-scheme (`http:`, `data:`) is not followed.
-
-What the bytes say beats what the manifest says: a JPEG listed as
-`image/png` comes back as `image/jpeg`, with a note. A cover larger than
-`bookmeta::MAX_COVER` (16 MiB) is left out with a note, in either format, and
-an EPUB's is never decompressed further than that, whatever size the archive
-claims for it.
+`Metadata::cover` is the cover image a Kindle book carries (EXTH 201, or the
+thumbnail, 202, when no cover is named), with its media type read off its
+first bytes. It is for library callers and is not part of the JSON record,
+which is a line of text.
 
 ## Asking a model
 

crates/bookmeta/src/epub.rs

@@ -4,17 +4,10 @@
 //! document carries Dublin Core elements: title, creator, publisher, date,
 //! language, identifier, subject. Calibre adds its own `<meta>` entries for
 //! the series, which nothing in the standard has a place for.
-//!
-//! The cover is whichever image the package names, asked in the order the
-//! standards came: EPUB 3's `cover-image` property, EPUB 2's
-//! `<meta name="cover">`, the guide's cover page and the image on it, and
-//! last any image whose name says cover. Every address is resolved inside the
-//! archive and nowhere else, and every member is read no further than a fixed
-//! bound, [`MAX_COVER`] for an image.
 
 use std::io::Read;
 
-use crate::{Cover, MAX_COVER, Metadata, Source, image_type};
+use crate::{Metadata, Source};
 
 /// How large a package document is worth reading.
 ///
@@ -45,7 +38,6 @@ pub fn read(path: &str, bytes: &[u8]) -> Metadata {
     };
 
     fill(&mut meta, &package);
-    meta.cover = cover(&mut archive, &package_path, &package, &mut meta);
     if meta.is_empty() {
         meta.note("a package document with nothing filled in");
     }
@@ -256,339 +248,6 @@ fn unescape(text: &str) -> String {
         .replace("&amp;", "&")
 }
 
-/// One entry of the package's manifest.
-struct Item {
-    id: String,
-    href: String,
-    media_type: String,
-    properties: String,
-    /// Where `href` leads inside the archive, when it leads anywhere.
-    path: Option<String>,
-}
-
-/// The cover image, by the first of the four routes that reaches one.
-///
-/// A route that names something unusable leaves a note and the next route is
-/// asked, so a book whose declared cover is missing can still have the one
-/// its file name gives away.
-fn cover<R: std::io::Read + std::io::Seek>(
-    archive: &mut zip::ZipArchive<R>,
-    package_path: &str,
-    package: &str,
-    meta: &mut Metadata,
-) -> Option<Cover> {
-    let base = directory_of(package_path);
-    let items: Vec<Item> = tags(package)
-        .filter(|(name, _)| *name == "item")
-        .map(|(_, tag)| {
-            let href = attr(tag, "href").unwrap_or_default();
-            Item {
-                id: attr(tag, "id").unwrap_or_default(),
-                path: resolve(base, &href),
-                href,
-                media_type: attr(tag, "media-type").unwrap_or_default(),
-                properties: attr(tag, "properties").unwrap_or_default(),
-            }
-        })
-        .collect();
-    let mut look = Look {
-        archive,
-        items: &items,
-        tried: Vec::new(),
-        meta,
-    };
-
-    // EPUB 3: the manifest item whose properties say so.
-    for item in items.iter().filter(|item| {
-        item.properties
-            .split_ascii_whitespace()
-            .any(|property| property == "cover-image")
-    }) {
-        if let Some(found) = look.image(base, &item.href) {
-            return Some(found);
-        }
-    }
-
-    // EPUB 2: `<meta name="cover" content="…">` names an item by its id, or
-    // in some converters' hands by its address.
-    if let Some(named) = tags(package)
-        .filter(|(name, _)| *name == "meta")
-        .find(|(_, tag)| attr(tag, "name").as_deref() == Some("cover"))
-        .and_then(|(_, tag)| attr(tag, "content"))
-        .map(|content| content.trim().to_string())
-        .filter(|content| !content.is_empty())
-    {
-        let as_path = resolve(base, &named);
-        let href = items
-            .iter()
-            .find(|item| item.id == named)
-            .or_else(|| {
-                items
-                    .iter()
-                    .find(|item| item.path.is_some() && item.path == as_path)
-            })
-            .map_or(named.as_str(), |item| item.href.as_str());
-        if let Some(found) = look.image(base, href) {
-            return Some(found);
-        }
-    }
-
-    // The guide's cover page, and the first image on it.
-    if let Some(page) = tags(package)
-        .filter(|(name, _)| *name == "reference")
-        .find(|(_, tag)| attr(tag, "type").is_some_and(|kind| kind.eq_ignore_ascii_case("cover")))
-        .and_then(|(_, tag)| attr(tag, "href"))
-    {
-        if let Some(found) = look.page(base, &page) {
-            return Some(found);
-        }
-    }
-
-    // Last: an image whose id or address says cover.
-    for item in items.iter().filter(|item| {
-        item.media_type.starts_with("image/")
-            && (item.id.to_lowercase().contains("cover")
-                || item.href.to_lowercase().contains("cover"))
-    }) {
-        if let Some(found) = look.image(base, &item.href) {
-            return Some(found);
-        }
-    }
-    None
-}
-
-/// The search for a cover, remembering what it has already looked at so
-/// that two routes to the same member read it, and note it, once.
-struct Look<'a, R> {
-    archive: &'a mut zip::ZipArchive<R>,
-    items: &'a [Item],
-    tried: Vec<String>,
-    meta: &'a mut Metadata,
-}
-
-impl<R: std::io::Read + std::io::Seek> Look<'_, R> {
-    /// Where an address leads, or a note saying it leads nowhere it may.
-    fn locate(&mut self, base: &str, href: &str) -> Option<String> {
-        let Some(path) = resolve(base, href) else {
-            self.meta.note(format!(
-                "the cover's address, {href}, is not a path inside the archive"
-            ));
-            return None;
-        };
-        if self.tried.contains(&path) {
-            return None;
-        }
-        self.tried.push(path.clone());
-        Some(path)
-    }
-
-    /// An image at an address, if that is what is there.
-    fn image(&mut self, base: &str, href: &str) -> Option<Cover> {
-        let path = self.locate(base, href)?;
-        let bytes = self.fetch(&path, MAX_COVER, "cover")?;
-        self.check(&path, bytes)
-    }
-
-    /// The image a cover page shows, or the page itself where the guide
-    /// points straight at an image.
-    fn page(&mut self, base: &str, href: &str) -> Option<Cover> {
-        let path = self.locate(base, href)?;
-        let bytes = self.fetch(&path, MAX_PACKAGE as usize, "cover page")?;
-        if image_type(&bytes).is_some() {
-            return self.check(&path, bytes);
-        }
-        let text = String::from_utf8_lossy(&bytes);
-        let Some(src) = tags(&text).find_map(|(name, tag)| match name {
-            "img" => attr(tag, "src"),
-            "image" => attr(tag, "xlink:href").or_else(|| attr(tag, "href")),
-            _ => None,
-        }) else {
-            self.meta
-                .note(format!("the cover page, {path}, shows no image"));
-            return None;
-        };
-        self.image(directory_of(&path), &src)
-    }
-
-    /// One member's bytes, read no further than `cap`.
-    ///
-    /// The size the archive declares is checked first and the read is
-    /// bounded anyway, because the declaration may be a lie.
-    fn fetch(&mut self, path: &str, cap: usize, what: &str) -> Option<Vec<u8>> {
-        let file = match self.archive.by_name(path) {
-            Ok(file) => file,
-            Err(zip::result::ZipError::FileNotFound) => {
-                self.meta
-                    .note(format!("the {what}, {path}, is not in the archive"));
-                return None;
-            }
-            Err(_) => {
-                self.meta
-                    .note(format!("the {what}, {path}, could not be read"));
-                return None;
-            }
-        };
-        let too_large = format!("the {what}, {path}, is larger than {}", crate::size(cap));
-        if file.size() > cap as u64 {
-            self.meta.note(too_large);
-            return None;
-        }
-        let mut bytes = Vec::new();
-        if file.take(cap as u64 + 1).read_to_end(&mut bytes).is_err() {
-            self.meta
-                .note(format!("the {what}, {path}, could not be read"));
-            return None;
-        }
-        if bytes.len() > cap {
-            self.meta.note(too_large);
-            return None;
-        }
-        Some(bytes)
-    }
-
-    /// The bytes as a cover when they are an image, typed by what they are
-    /// rather than by what the manifest says.
-    fn check(&mut self, path: &str, bytes: Vec<u8>) -> Option<Cover> {
-        let Some(media_type) = image_type(&bytes) else {
-            self.meta
-                .note(format!("the cover, {path}, is not an image"));
-            return None;
-        };
-        if let Some(item) = self
-            .items
-            .iter()
-            .find(|item| item.path.as_deref() == Some(path))
-        {
-            let declared = item.media_type.trim();
-            if !declared.is_empty() && !declared.eq_ignore_ascii_case(media_type) {
-                self.meta.note(format!(
-                    "the manifest calls the cover, {path}, {declared}, but it is {media_type}"
-                ));
-            }
-        }
-        Some(Cover {
-            media_type: media_type.to_string(),
-            bytes,
-        })
-    }
-}
-
-/// The directory a member sits in, with no trailing slash.
-fn directory_of(path: &str) -> &str {
-    path.rsplit_once('/').map_or("", |(dir, _)| dir)
-}
-
-/// Where an address written in a member in `base` leads inside the archive.
-///
-/// The fragment comes off and percent-escapes are undone. An address with a
-/// scheme, or one that climbs above the root of the archive, leads nowhere
-/// this will follow.
-fn resolve(base: &str, href: &str) -> Option<String> {
-    let href = href.split('#').next().unwrap_or("").trim();
-    // `http:`, `data:`, `file:`: a scheme before the first slash.
-    if let Some(colon) = href.find(':') {
-        if !href[..colon].contains('/') {
-            return None;
-        }
-    }
-    let href = percent_decode(href)?;
-    let mut parts: Vec<&str> = if href.starts_with('/') {
-        Vec::new()
-    } else {
-        base.split('/').filter(|part| !part.is_empty()).collect()
-    };
-    for part in href.split('/') {
-        match part {
-            "" | "." => {}
-            ".." => {
-                parts.pop()?;
-            }
-            part => parts.push(part),
-        }
-    }
-    // An address of nothing but the directory it is written in names no
-    // member at all.
-    if href.is_empty() || parts.is_empty() {
-        return None;
-    }
-    Some(parts.join("/"))
-}
-
-/// `%20` back to a space, and so on, where the result is still UTF-8.
-fn percent_decode(text: &str) -> Option<String> {
-    let raw = text.as_bytes();
-    let mut out = Vec::with_capacity(raw.len());
-    let mut at = 0;
-    while at < raw.len() {
-        if raw[at] == b'%' {
-            let hex = std::str::from_utf8(raw.get(at + 1..at + 3)?).ok()?;
-            out.push(u8::from_str_radix(hex, 16).ok()?);
-            at += 3;
-        } else {
-            out.push(raw[at]);
-            at += 1;
-        }
-    }
-    String::from_utf8(out).ok()
-}
-
-/// Every start tag in a document, in order, as its local name and its text.
-///
-/// Comments, declarations, processing instructions and end tags are passed
-/// over. The local name is what follows any namespace prefix, so
-/// `<opf:item>` is an `item`.
-fn tags(xml: &str) -> impl Iterator<Item = (&str, &str)> {
-    let mut rest = xml;
-    std::iter::from_fn(move || {
-        loop {
-            let at = rest.find('<')?;
-            rest = &rest[at..];
-            if let Some(after) = rest.strip_prefix("<!--") {
-                rest = after.find("-->").map_or("", |end| &after[end + 3..]);
-                continue;
-            }
-            let end = rest.find('>')?;
-            let tag = &rest[1..end];
-            rest = &rest[end + 1..];
-            if tag.starts_with(['/', '!', '?']) {
-                continue;
-            }
-            let name = tag
-                .split(|c: char| c.is_ascii_whitespace() || c == '/')
-                .next()
-                .unwrap_or("");
-            let local = name.rsplit_once(':').map_or(name, |(_, local)| local);
-            return Some((local, tag));
-        }
-    })
-}
-
-/// The value of one attribute in a tag's text, entities undone.
-///
-/// The name has to stand on its own — `id` is not the end of `xml:id` or
-/// `data-id` — and may have space either side of its `=`.
-fn attr(tag: &str, name: &str) -> Option<String> {
-    let mut from = 0;
-    while let Some(found) = tag[from..].find(name) {
-        let at = from + found;
-        from = at + name.len();
-        if !tag[..at].ends_with(|c: char| c.is_ascii_whitespace()) {
-            continue;
-        }
-        let Some(rest) = tag[from..].trim_start().strip_prefix('=') else {
-            continue;
-        };
-        let rest = rest.trim_start();
-        let Some(quote) = rest.chars().next().filter(|c| matches!(c, '"' | '\'')) else {
-            continue;
-        };
-        let value = &rest[1..];
-        let end = value.find(quote)?;
-        return Some(unescape(&value[..end]));
-    }
-    None
-}
-
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -697,386 +356,4 @@ mod tests {
         assert_eq!(meta.source, "none");
         assert!(!meta.notes.is_empty());
     }
-
-    // Covers. Each book is built here, a few members zipped in memory: the
-    // container, a package in `OEBPS/`, and whatever the case is about.
-
-    const JPEG: &[u8] = b"\xff\xd8\xff\xe0\x00\x10JFIF\x00jpeg body";
-    const PNG: &[u8] = b"\x89PNG\r\n\x1a\n\x00\x00\x00\x0dIHDR png body";
-    const GIF: &[u8] = b"GIF89a\x01\x00\x01\x00 gif body";
-    const WEBP: &[u8] = b"RIFF\x20\x00\x00\x00WEBPVP8 webp body";
-
-    const CONTAINER: &str = r#"<?xml version="1.0"?>
-<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">
-  <rootfiles>
-    <rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/>
-  </rootfiles>
-</container>"#;
-
-    /// A package with this manifest and these extra elements, around a title.
-    fn opf(manifest: &str, extra: &str) -> String {
-        format!(
-            r#"<?xml version="1.0"?>
-<package xmlns="http://www.idpf.org/2007/opf" version="3.0">
-  <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
-    <dc:title>Covered</dc:title>
-  </metadata>
-  <manifest>
-    <item id="text" href="text/one.xhtml" media-type="application/xhtml+xml"/>
-    {manifest}
-  </manifest>
-  {extra}
-</package>"#
-        )
-    }
-
-    /// An EPUB of these members, after the container.
-    fn epub(members: &[(&str, &[u8])]) -> Vec<u8> {
-        use std::io::Write;
-        let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
-        let options = zip::write::SimpleFileOptions::default()
-            .compression_method(zip::CompressionMethod::Deflated);
-        let container: (&str, &[u8]) = ("META-INF/container.xml", CONTAINER.as_bytes());
-        for (name, data) in std::iter::once(&container).chain(members) {
-            zip.start_file(*name, options).unwrap();
-            zip.write_all(data).unwrap();
-        }
-        zip.finish().unwrap().into_inner()
-    }
-
-    fn book(package: &str, members: &[(&str, &[u8])]) -> Metadata {
-        let mut all = vec![("OEBPS/content.opf", package.as_bytes())];
-        all.extend_from_slice(members);
-        read("x.epub", &epub(&all))
-    }
-
-    #[test]
-    fn epub_3_names_its_cover_with_a_property() {
-        let package = opf(
-            r#"<item id="decoy-cover" href="images/cover-thumb.png" media-type="image/png"/>
-               <item id="c" href="images/front.jpg" media-type="image/jpeg" properties="svg cover-image"/>"#,
-            r#"<metadata><meta name="cover" content="decoy-cover"/></metadata>"#,
-        );
-        let meta = book(
-            &package,
-            &[
-                ("OEBPS/images/front.jpg", JPEG),
-                ("OEBPS/images/cover-thumb.png", PNG),
-            ],
-        );
-        let cover = meta.cover.expect("a cover");
-        assert_eq!(cover.media_type, "image/jpeg");
-        assert_eq!(cover.bytes, JPEG);
-        assert!(meta.notes.is_empty(), "{:?}", meta.notes);
-    }
-
-    #[test]
-    fn epub_2_names_its_cover_with_a_meta_holding_an_id() {
-        let package = opf(
-            r#"<opf:item id="cover-img" href="Images/Front%20Page.png#top" media-type="image/png"/>"#,
-            r#"<metadata><meta content="cover-img" name="cover" /></metadata>"#,
-        );
-        let meta = book(&package, &[("OEBPS/Images/Front Page.png", PNG)]);
-        let cover = meta.cover.expect("a cover");
-        assert_eq!(cover.media_type, "image/png");
-        assert_eq!(cover.bytes, PNG);
-        assert!(meta.notes.is_empty(), "{:?}", meta.notes);
-    }
-
-    #[test]
-    fn a_cover_meta_holding_an_address_is_followed_as_one() {
-        // Listed in the manifest under another id, and not listed at all.
-        let listed = opf(
-            r#"<item id="x1" href="../art/front.gif" media-type="image/gif"/>"#,
-            r#"<metadata><meta name="cover" content="../art/front.gif"/></metadata>"#,
-        );
-        let meta = book(&listed, &[("art/front.gif", GIF)]);
-        assert_eq!(meta.cover.expect("a cover").media_type, "image/gif");
-
-        let unlisted = opf(
-            "",
-            r#"<metadata><meta name="cover" content="front.webp"/></metadata>"#,
-        );
-        let meta = book(&unlisted, &[("OEBPS/front.webp", WEBP)]);
-        let cover = meta.cover.expect("a cover");
-        assert_eq!(cover.media_type, "image/webp");
-        assert_eq!(cover.bytes, WEBP);
-    }
-
-    #[test]
-    fn the_guide_cover_page_gives_the_first_image_on_it() {
-        let package = opf(
-            r#"<item id="page" href="text/cover.xhtml" media-type="application/xhtml+xml"/>
-               <item id="art" href="images/art.jpg" media-type="image/jpeg"/>"#,
-            r#"<guide><reference type="Cover" title="Cover" href="text/cover.xhtml#start"/></guide>"#,
-        );
-        let page = br#"<html><body><!-- <img src="nothing.png"/> -->
-            <img alt="cover" src="../images/art.jpg"/><img src="../images/later.png"/></body></html>"#;
-        let meta = book(
-            &package,
-            &[
-                ("OEBPS/text/cover.xhtml", page),
-                ("OEBPS/images/art.jpg", JPEG),
-                ("OEBPS/images/later.png", PNG),
-            ],
-        );
-        let cover = meta.cover.expect("a cover");
-        assert_eq!(cover.bytes, JPEG);
-        assert!(meta.notes.is_empty(), "{:?}", meta.notes);
-    }
-
-    #[test]
-    fn the_guide_cover_page_may_draw_its_image_in_svg() {
-        let package = opf(
-            "",
-            r#"<guide><reference type="cover" href="titlepage.xhtml"/></guide>"#,
-        );
-        for image in [
-            r#"<svg:image width="600" xlink:href="cover.png"/>"#,
-            r#"<image href="cover.png" />"#,
-        ] {
-            let page = format!(r#"<html><body><svg:svg>{image}</svg:svg></body></html>"#);
-            let meta = book(
-                &package,
-                &[
-                    ("OEBPS/titlepage.xhtml", page.as_bytes()),
-                    ("OEBPS/cover.png", PNG),
-                ],
-            );
-            assert_eq!(meta.cover.expect(image).bytes, PNG);
-        }
-    }
-
-    #[test]
-    fn last_an_image_whose_name_says_cover() {
-        let package = opf(
-            r#"<item id="pic1" href="images/plate.jpg" media-type="image/jpeg"/>
-               <item id="CoverImage" href="images/i2.gif" media-type="image/gif"/>"#,
-            "",
-        );
-        let meta = book(
-            &package,
-            &[
-                ("OEBPS/images/plate.jpg", JPEG),
-                ("OEBPS/images/i2.gif", GIF),
-            ],
-        );
-        assert_eq!(meta.cover.expect("a cover").bytes, GIF);
-
-        let by_address = opf(
-            r#"<item id="i9" href="images/Book_COVER.jpeg" media-type="image/jpeg"/>"#,
-            "",
-        );
-        let meta = book(&by_address, &[("OEBPS/images/Book_COVER.jpeg", JPEG)]);
-        assert_eq!(meta.cover.expect("a cover").bytes, JPEG);
-    }
-
-    #[test]
-    fn a_book_that_names_no_cover_has_none_and_says_nothing() {
-        let package = opf(
-            r#"<item id="plate" href="images/plate.jpg" media-type="image/jpeg"/>"#,
-            "",
-        );
-        let meta = book(&package, &[("OEBPS/images/plate.jpg", JPEG)]);
-        assert_eq!(meta.cover, None);
-        assert!(meta.notes.is_empty(), "{:?}", meta.notes);
-    }
-
-    #[test]
-    fn a_mislabelled_cover_is_what_its_bytes_say() {
-        let package = opf(
-            r#"<item id="c" href="c.png" media-type="image/png" properties="cover-image"/>"#,
-            "",
-        );
-        let meta = book(&package, &[("OEBPS/c.png", JPEG)]);
-        assert_eq!(meta.cover.expect("a cover").media_type, "image/jpeg");
-        assert_eq!(
-            meta.notes,
-            ["the manifest calls the cover, OEBPS/c.png, image/png, but it is image/jpeg"]
-        );
-    }
-
-    #[test]
-    fn a_cover_that_is_not_an_image_is_a_note_and_not_a_cover() {
-        let package = opf(
-            r#"<item id="c" href="c.jpg" media-type="image/jpeg" properties="cover-image"/>"#,
-            "",
-        );
-        let meta = book(&package, &[("OEBPS/c.jpg", b"<html>not an image</html>")]);
-        assert_eq!(meta.cover, None);
-        assert_eq!(meta.notes, ["the cover, OEBPS/c.jpg, is not an image"]);
-    }
-
-    #[test]
-    fn a_cover_larger_than_the_cap_is_left_out() {
-        let mut huge = JPEG.to_vec();
-        huge.resize(MAX_COVER + 1, 0);
-        let package = opf(
-            r#"<item id="c" href="c.jpg" media-type="image/jpeg" properties="cover-image"/>"#,
-            "",
-        );
-        let meta = book(&package, &[("OEBPS/c.jpg", &huge)]);
-        assert_eq!(meta.cover, None);
-        assert_eq!(
-            meta.notes,
-            ["the cover, OEBPS/c.jpg, is larger than 16 MiB"]
-        );
-    }
-
-    /// Read one member through `Look::fetch` with a cap of `cap`.
-    fn fetch(bytes: Vec<u8>, cap: usize) -> (Option<usize>, Vec<String>) {
-        let mut meta = Metadata::empty("x.epub", Source::Opf);
-        let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
-        let mut look = Look {
-            archive: &mut archive,
-            items: &[],
-            tried: Vec::new(),
-            meta: &mut meta,
-        };
-        let got = look.fetch("big.jpg", cap, "cover").map(|bytes| bytes.len());
-        (got, meta.notes)
-    }
-
-    #[test]
-    fn a_member_is_read_no_further_than_its_cap() {
-        let bytes = epub(&[("big.jpg", &[0u8; 4096])]);
-        assert_eq!(fetch(bytes.clone(), 4096).0, Some(4096));
-        assert_eq!(
-            fetch(bytes, 4095),
-            (
-                None,
-                vec!["the cover, big.jpg, is larger than 4095 bytes".to_string()]
-            )
-        );
-    }
-
-    #[test]
-    fn a_member_whose_archive_understates_its_size_is_still_bounded() {
-        let mut bytes = epub(&[("big.jpg", &[0u8; 4096])]);
-        // Declare 10 bytes uncompressed, in the local header and in the
-        // central directory, where 4096 are compressed in.
-        // The name comes 30 bytes into a local header and 46 into a central
-        // directory entry, and the uncompressed size 22 and 24.
-        let local = find(&bytes, b"big.jpg") - 30;
-        bytes[local + 22..local + 26].copy_from_slice(&10u32.to_le_bytes());
-        let central = local + 31 + find(&bytes[local + 31..], b"big.jpg") - 46;
-        assert_eq!(&bytes[central..central + 4], b"PK\x01\x02");
-        bytes[central + 24..central + 28].copy_from_slice(&10u32.to_le_bytes());
-
-        // The declaration passes the check, and the read stops at the cap.
-        let (got, notes) = fetch(bytes, 100);
-        assert_eq!(got, None);
-        assert_eq!(notes, ["the cover, big.jpg, is larger than 100 bytes"]);
-    }
-
-    fn find(haystack: &[u8], needle: &[u8]) -> usize {
-        haystack
-            .windows(needle.len())
-            .position(|window| window == needle)
-            .expect("present")
-    }
-
-    #[test]
-    fn a_cover_named_but_missing_is_a_note_and_the_next_route_is_asked() {
-        let package = opf(
-            r#"<item id="c" href="gone.jpg" media-type="image/jpeg" properties="cover-image"/>"#,
-            "",
-        );
-        let meta = book(&package, &[]);
-        assert_eq!(meta.cover, None);
-        assert_eq!(
-            meta.notes,
-            ["the cover, OEBPS/gone.jpg, is not in the archive"]
-        );
-
-        // A missing page is said the same way.
-        let page = opf(
-            "",
-            r#"<guide><reference type="cover" href="none.xhtml"/></guide>"#,
-        );
-        assert_eq!(
-            book(&page, &[]).notes,
-            ["the cover page, OEBPS/none.xhtml, is not in the archive"]
-        );
-
-        // The same file reached again by another route is not read twice,
-        // and an image the name gives away still stands in.
-        let package = opf(
-            r#"<item id="cover" href="gone.jpg" media-type="image/jpeg" properties="cover-image"/>
-               <item id="cover-small" href="small.png" media-type="image/png"/>"#,
-            r#"<metadata><meta name="cover" content="cover"/></metadata>"#,
-        );
-        let meta = book(&package, &[("OEBPS/small.png", PNG)]);
-        assert_eq!(meta.cover.expect("the stand-in").bytes, PNG);
-        assert_eq!(
-            meta.notes,
-            ["the cover, OEBPS/gone.jpg, is not in the archive"]
-        );
-    }
-
-    #[test]
-    fn an_address_never_leaves_the_archive() {
-        assert_eq!(resolve("OEBPS", "../c.jpg").as_deref(), Some("c.jpg"));
-        assert_eq!(resolve("OEBPS", "/c.jpg").as_deref(), Some("c.jpg"));
-        assert_eq!(
-            resolve("OEBPS/text", "./a/../b.jpg").as_deref(),
-            Some("OEBPS/text/b.jpg")
-        );
-        assert_eq!(resolve("", "a%2Fb.jpg").as_deref(), Some("a/b.jpg"));
-        for hostile in [
-            "../../c.jpg",
-            "../../../etc/passwd",
-            "http://example.com/c.jpg",
-            "data:image/png;base64,AAAA",
-            "file:///etc/passwd",
-            "%ZZ.jpg",
-            "%ff.jpg",
-            "c%",
-            "#only-a-fragment",
-            "",
-        ] {
-            assert_eq!(resolve("OEBPS", hostile), None, "{hostile}");
-        }
-
-        let package = opf(
-            r#"<item id="c" href="../../c.jpg" media-type="image/jpeg" properties="cover-image"/>"#,
-            "",
-        );
-        let meta = book(&package, &[("c.jpg", JPEG)]);
-        assert_eq!(meta.cover, None);
-        assert_eq!(
-            meta.notes,
-            ["the cover's address, ../../c.jpg, is not a path inside the archive"]
-        );
-    }
-
-    #[test]
-    fn an_attribute_is_matched_by_its_whole_name() {
-        let tag = r#"item data-id="no" xml:id="no" id = 'yes' href="a&amp;b.jpg""#;
-        assert_eq!(attr(tag, "id").as_deref(), Some("yes"));
-        assert_eq!(attr(tag, "href").as_deref(), Some("a&b.jpg"));
-        assert_eq!(attr(r#"image xlink:href="x.png""#, "href"), None);
-        assert_eq!(attr(r#"item id="unterminated"#, "id"), None);
-    }
-
-    #[test]
-    fn every_truncation_of_a_book_with_a_cover_returns() {
-        let package = opf(
-            r#"<item id="c" href="c.jpg" media-type="image/jpeg" properties="cover-image"/>"#,
-            r#"<guide><reference type="cover" href="p.xhtml"/></guide>"#,
-        );
-        let bytes = epub(&[
-            ("OEBPS/content.opf", package.as_bytes()),
-            ("OEBPS/c.jpg", JPEG),
-            ("OEBPS/p.xhtml", b"<img src=\"c.jpg\"/>"),
-        ]);
-        for len in 0..=bytes.len() {
-            let _ = read("cut.epub", &bytes[..len]);
-        }
-        for at in 0..bytes.len() {
-            let mut broken = bytes.clone();
-            broken[at] ^= 0xff;
-            let _ = read("broken.epub", &broken);
-        }
-    }
 }

crates/bookmeta/src/lib.rs

@@ -106,47 +106,12 @@ pub struct Metadata {
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct Cover {
     /// What kind of image it is, read off its first bytes: `image/jpeg`,
-    /// `image/png`, `image/gif`, `image/webp` or `image/bmp`.
+    /// `image/png`, `image/gif` or `image/bmp`.
     pub media_type: String,
-    /// The image itself, never more than [`MAX_COVER`] bytes.
+    /// The image itself.
     pub bytes: Vec<u8>,
 }
 
-/// How large a cover is worth keeping.
-///
-/// A real one is a few hundred kilobytes. An EPUB's cover has to be
-/// decompressed to be read, and this is as far as that goes: a larger one is
-/// left out with a note, in every format, so a hostile file cannot make a
-/// reader allocate whatever it likes.
-pub const MAX_COVER: usize = 16 * 1024 * 1024;
-
-/// A size, as a note says it.
-pub(crate) fn size(bytes: usize) -> String {
-    const MIB: usize = 1024 * 1024;
-    if bytes >= MIB && bytes % MIB == 0 {
-        format!("{} MiB", bytes / MIB)
-    } else {
-        format!("{bytes} bytes")
-    }
-}
-
-/// What kind of image these bytes are, by their magic.
-pub(crate) fn image_type(data: &[u8]) -> Option<&'static str> {
-    if data.starts_with(&[0xff, 0xd8, 0xff]) {
-        Some("image/jpeg")
-    } else if data.starts_with(b"\x89PNG\r\n\x1a\n") {
-        Some("image/png")
-    } else if data.starts_with(b"GIF87a") || data.starts_with(b"GIF89a") {
-        Some("image/gif")
-    } else if data.starts_with(b"RIFF") && data.get(8..12) == Some(b"WEBP") {
-        Some("image/webp")
-    } else if data.starts_with(b"BM") && data.len() > 14 {
-        Some("image/bmp")
-    } else {
-        None
-    }
-}
-
 fn version() -> u32 {
     corpus_core::V
 }

crates/bookmeta/src/mobi.rs

@@ -13,7 +13,7 @@
 //! panics or reads past the end.
 
 use crate::epub::{digits_of, is_isbn, is_uuid};
-use crate::{Cover, MAX_COVER, Metadata, Source, image_type};
+use crate::{Cover, Metadata, Source};
 
 /// Where a Palm database says what it holds.
 const TYPE_AT: usize = 60;
@@ -332,13 +332,6 @@ fn cover(
         meta.note("the cover record is not an image");
         return None;
     };
-    if data.len() > MAX_COVER {
-        meta.note(format!(
-            "the cover record is larger than {}",
-            crate::size(MAX_COVER)
-        ));
-        return None;
-    }
     Some(Cover {
         media_type: media_type.to_string(),
         bytes: data.to_vec(),
@@ -354,9 +347,6 @@ fn cover(
 fn first_image_record(bytes: &[u8], records: &[usize], first_image: Option<u32>) -> Option<Cover> {
     let first = first_image.filter(|&index| index != NONE)?;
     let data = record(bytes, records, first as usize)?;
-    if data.len() > MAX_COVER {
-        return None;
-    }
     let media_type = image_type(data).filter(|kind| *kind != "image/bmp")?;
     Some(Cover {
         media_type: media_type.to_string(),
@@ -369,6 +359,21 @@ fn is_asin(value: &str) -> bool {
     value.len() == 10 && value.chars().all(|c| c.is_ascii_alphanumeric())
 }
 
+/// What kind of image these bytes are, by their magic.
+fn image_type(data: &[u8]) -> Option<&'static str> {
+    if data.starts_with(&[0xff, 0xd8, 0xff]) {
+        Some("image/jpeg")
+    } else if data.starts_with(b"\x89PNG\r\n\x1a\n") {
+        Some("image/png")
+    } else if data.starts_with(b"GIF87a") || data.starts_with(b"GIF89a") {
+        Some("image/gif")
+    } else if data.starts_with(b"BM") && data.len() > 14 {
+        Some("image/bmp")
+    } else {
+        None
+    }
+}
+
 /// The Palm database's name, the last place a title can come from.
 ///
 /// Converters write the title there with spaces made underscores; it is

docs/contract.md

@@ -64,7 +64,7 @@ No network, no database, no configuration file, no state kept between runs.
 Five exceptions, each declared in `--help`:
 
 - [`emb`](../crates/emb/README.md) makes an HTTP call, which is the whole of what it does.
-- [`bibfetch`](../crates/bibfetch/README.md) asks a register about an identifier, likewise.
+- [`bibfetch`](../crates/bibfetch/README.md) asks a register about an identifier or a title, likewise.
 - [`pdftriage`](../crates/pdftriage/README.md) runs `pdftotext` and `pdfinfo`.
 - [`pdfgutter`](../crates/pdfgutter/README.md) runs `pdftoppm` to look at a page.
 - [`avplan`](../crates/avplan/README.md) runs `ffprobe` to read a file's streams, and

docs/decisions.md

@@ -20,7 +20,6 @@ to any of them is a change to a decision rather than to an accident.
 - [pdfgutter — a page tree that does not add up is refused](#pdfgutter--a-page-tree-that-does-not-add-up-is-refused)
 - [pdfgutter — a sheet on its side is cut the way it is read](#pdfgutter--a-sheet-on-its-side-is-cut-the-way-it-is-read)
 - [pdfgutter — an unreadable file is an answer, not a false](#pdfgutter--an-unreadable-file-is-an-answer-not-a-false)
-- [bookmeta — a cover is what its bytes say it is](#bookmeta--a-cover-is-what-its-bytes-say-it-is)
 - [Every pattern — two regex engines, two Unicode tables](#every-pattern--two-regex-engines-two-unicode-tables)
 
 ## cuenorm — settings after the end time are the format, not a malformation
@@ -244,23 +243,6 @@ answers, and the [contract](contract.md) gives them three different numbers. A
 caller wanting them collapsed treats 1 and 2 alike; a caller wanting to know
 which files it failed on can now find out.
 
-## bookmeta — a cover is what its bytes say it is
-
-An EPUB's manifest gives every image a media type, and the cover's is not
-used: the type comes from the image's first bytes, and a disagreement is a
-note. Elsewhere `bookmeta` reports what the file says rather than what is
-true, and this is the one place it does not.
-
-The difference is that a title nobody believes is still a title, while a
-media type that is wrong is an image that does not open. A caller handed
-`image/png` for a JPEG writes `cover.png`, serves it as PNG, or hands it to a
-decoder that refuses it. An image can be replaced by a converter or by hand
-without its manifest entry changing, and the note keeps what the file said,
-so nothing is lost by correcting it.
-
-The same reasoning goes the other way: bytes that are not an image are not a
-cover, whatever the manifest calls them.
-
 ## Every pattern — two regex engines, two Unicode tables
 
 Patterns here spell their character classes explicitly rather than leaning on