bookmeta: an EPUB cover by the word, not the letters

The last route took any image whose id or address contained "cover", so
images/discover-ch1.png listed before images/cover.jpg won, and
recovery-map.png alone became the cover. A name now says cover only with
it as a word of its own: words break at anything not a letter or digit
and where a lower case letter meets a capital, so CoverImage and
Book_COVER still do. An image named cover outright comes first, then
manifest order. It is still a guess.

The guide's cover page took its first image, which can be a logo. Where
the page has several, the one whose file name, alt, id or class says
cover is taken; where none does, the first is, with a note saying how
many there were.

Each member was read no further than MAX_COVER, but nothing bounded how
many: two hundred cover-named images of 16 MiB took seven seconds to
inflate and left two hundred notes. One search now reads at most eight
members and stops with one note.

A percent-escape takes two hex digits and nothing else. from_str_radix
also takes a sign, so a%+fb decoded to a, 0x0f, b.

crates/bibfetch/README.md

@@ -136,6 +136,14 @@ redirect included, stops after two minutes. The two minutes start once it is
 the fetch's turn, so a fetch queued behind others in the same process can wait
 that long again first.
 
+Open Library allows a hundred covers by ISBN every five minutes from one
+address, and refuses the rest with `403`. Covers asked by ISBN are kept three
+seconds apart across the whole process (`Covers::pace`), which stays under that
+limit however long a run goes. A refusal all the same is not mistaken for an
+outage: the other form of the ISBN is not asked, the cover id is, because
+Open Library does not limit those, and with no cover found that way the fetch
+fails with `Error::Limited`, which means ask again later.
+
 | Option | Effect |
 |---|---|
 | `-o`, `--output <FILE>` | Where to write the image |
@@ -144,7 +152,8 @@ that long again first.
 
 Exit `0` the cover was written, `1` Open Library has none, `65` the argument
 was not an ISBN or the answer was not an image, `69` Open Library could not be
-reached, `74` the file could not be written.
+reached, `74` the file could not be written, `75` Open Library refused for
+being asked too often and will answer later.
 
 From Rust, `bibfetch::cover(&agent, isbn, cover_id, size)` returns
 `Ok(None)` when there is no cover, and a [`Cover`](src/cover.rs) with the

crates/bibfetch/src/cover.rs

@@ -46,6 +46,11 @@ const REDIRECT_HOSTS: [&str; 2] = ["covers.openlibrary.org", "archive.org"];
 /// agent's timeout each, and a minute's `Retry-After` between attempts.
 pub const DEADLINE: Duration = Duration::from_secs(120);
 
+/// How long to leave between two covers asked by ISBN. Open Library allows a
+/// hundred every five minutes from one address and refuses the rest with
+/// `403`; one every three seconds stays under that however long a run goes.
+pub const PACE: Duration = Duration::from_secs(3);
+
 /// The longest a `Retry-After` is honoured for. A register asking for more
 /// than this is better treated as down than waited on.
 const MAX_WAIT: Duration = Duration::from_secs(60);
@@ -54,7 +59,10 @@ const MAX_WAIT: Duration = Duration::from_secs(60);
 /// this, and limits covers asked by ISBN to a hundred every five minutes
 /// from one address; threads asking at once would only meet the limit
 /// sooner and all back off together.
-static ONE_AT_A_TIME: Mutex<()> = Mutex::new(());
+///
+/// It holds when the last cover was asked for by ISBN, so that [`PACE`] is
+/// kept across fetches and threads.
+static ONE_AT_A_TIME: Mutex<Option<Instant>> = Mutex::new(None);
 
 /// Which of Open Library's three sizes to ask for.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
@@ -127,6 +135,9 @@ pub struct Covers {
     /// request is given whatever is left of it, or the agent's own timeout
     /// if that is shorter, and a wait that would run past it is not waited.
     pub deadline: Duration,
+    /// The least time between two covers asked by ISBN; [`PACE`] by default.
+    /// Covers asked by id are not limited and are not spaced out.
+    pub pace: Duration,
 }
 
 impl Default for Covers {
@@ -135,6 +146,7 @@ impl Default for Covers {
             base: OPEN_LIBRARY_COVERS.to_string(),
             attempts: crate::ATTEMPTS,
             deadline: DEADLINE,
+            pace: PACE,
         }
     }
 }
@@ -162,6 +174,12 @@ impl Covers {
     /// of the ISBN does not stop the other being asked, but it does stop the
     /// cover id being asked: the ISBN's own cover may exist, and is the one
     /// wanted. The failure is returned when no ISBN form gave a cover.
+    ///
+    /// Refusal for asking too often is the exception. It is
+    /// [`Error::Limited`], the other form is not asked, since it would only
+    /// be refused too, and the cover id is still asked, because Open Library
+    /// does not limit those. A cover found that way is returned; otherwise the
+    /// refusal is, so that the caller knows to ask again later.
     pub fn fetch(
         &self,
         agent: &ureq::Agent,
@@ -186,31 +204,54 @@ impl Covers {
             .map(|form| format!("{}/b/isbn/{form}-{letter}.jpg?default=false", self.base))
             .collect();
 
-        let _turn = ONE_AT_A_TIME
+        let mut last_by_isbn = ONE_AT_A_TIME
             .lock()
             .unwrap_or_else(|held| held.into_inner());
         // From here, not from the call: waiting for the lock is time another
         // fetch spent, and counting it would let a busy process starve this
-        // one of the time its own requests need.
+        // one of the time its own requests need. A deadline too long to add
-        let until = Instant::now() + self.deadline;
+        // to the clock is no deadline at all.
+        let now = Instant::now();
+        let until = now
+            .checked_add(self.deadline)
+            .unwrap_or_else(|| now + Duration::from_secs(365 * 24 * 60 * 60));
         let mut failed = None;
         for address in &addresses {
-            match self.ask(agent, address, until) {
+            if let Some(ready) = last_by_isbn.and_then(|at| at.checked_add(self.pace)) {
+                let now = Instant::now();
+                if ready > now {
+                    if ready >= until {
+                        return Err(Error::Unreachable(format!(
+                            "{address}: past the {:?} deadline, waiting to keep {:?} \
+between covers asked by ISBN",
+                            self.deadline, self.pace
+                        )));
+                    }
+                    std::thread::sleep(ready - now);
+                }
+            }
+            let answer = self.ask(agent, address, until);
+            *last_by_isbn = Some(Instant::now());
+            match answer {
                 Ok(Answer::Image(cover)) => return Ok(Some(cover)),
                 Ok(Answer::Absent) => {}
+                Err(why @ Error::Limited(_)) => {
+                    failed = Some(why);
+                    break;
+                }
                 Err(why) => failed = Some(why),
             }
         }
-        if let Some(why) = failed {
+        if let Some(why) = failed.take_if(|why| !matches!(why, Error::Limited(_))) {
             return Err(why);
         }
         let Some(id) = cover_id else {
-            return Ok(None);
+            return failed.map_or(Ok(None), Err);
         };
         let address = format!("{}/b/id/{id}-{letter}.jpg?default=false", self.base);
         match self.ask(agent, &address, until)? {
             Answer::Image(cover) => Ok(Some(cover)),
-            Answer::Absent => Ok(None),
+            Answer::Absent => failed.map_or(Ok(None), Err),
         }
     }
 
@@ -311,6 +352,15 @@ Open Library's covers or the Internet Archive"
                 // `default=false` makes a missing cover a 404 rather than a
                 // stand-in image.
                 404 | 410 => return Ok(Answer::Absent),
+                // Open Library's answer to more than a hundred covers asked by
+                // ISBN in five minutes from one address. Asking again at once
+                // would only be refused again.
+                403 => {
+                    return Err(Fault::Final(Error::Limited(format!(
+                        "{at}: refused (403); Open Library allows a hundred covers \
+by ISBN every five minutes from one address"
+                    ))));
+                }
                 429 | 503 => {
                     let wait = header(&response, "retry-after")
                         .and_then(|value| value.trim().parse::<u64>().ok())
@@ -603,6 +653,7 @@ mod tests {
                 base: self.base.clone(),
                 attempts: 3,
                 deadline: DEADLINE,
+                pace: Duration::ZERO,
             }
         }
 
@@ -1052,6 +1103,73 @@ mod tests {
         assert_eq!(stub.asked(), [ISBN13, ISBN10]);
     }
 
+    #[test]
+    fn a_refusal_for_asking_too_often_still_asks_the_cover_id() {
+        let stub = Stub::start(|_, path| match path {
+            ISBN13 => Reply::status(403),
+            "/b/id/42-L.jpg?default=false" => Reply::image("image/jpeg", jpeg(4096)),
+            _ => Reply::status(404),
+        });
+        let cover = stub
+            .covers()
+            .fetch(&agent(), "9780306406157", Some(42), CoverSize::Large)
+            .expect("an answer")
+            .expect("a cover");
+        assert!(cover.url.ends_with("/b/id/42-L.jpg?default=false"));
+        // The ten-digit form would only have been refused too.
+        assert_eq!(stub.asked(), [ISBN13, "/b/id/42-L.jpg?default=false"]);
+    }
+
+    #[test]
+    fn a_refusal_for_asking_too_often_is_not_an_absence() {
+        let stub = Stub::start(|_, path| match path {
+            ISBN13 => Reply::status(403),
+            _ => Reply::status(404),
+        });
+        let found = stub
+            .covers()
+            .fetch(&agent(), "9780306406157", None, CoverSize::Large)
+            .map(|cover| cover.map(|cover| cover.url));
+        assert!(matches!(found, Err(Error::Limited(_))), "{found:?}");
+        assert_eq!(stub.asked(), [ISBN13]);
+
+        // With a cover id that has no cover either, the refusal still stands:
+        // the ISBN's own cover may be there once the limit passes.
+        let found = stub
+            .covers()
+            .fetch(&agent(), "9780306406157", Some(42), CoverSize::Large)
+            .map(|cover| cover.map(|cover| cover.url));
+        assert!(matches!(found, Err(Error::Limited(_))), "{found:?}");
+    }
+
+    #[test]
+    fn a_deadline_too_long_to_count_is_no_deadline() {
+        let stub = Stub::start(|_, _| Reply::image("image/jpeg", jpeg(2048)));
+        let covers = Covers {
+            deadline: Duration::MAX,
+            ..stub.covers()
+        };
+        let cover = covers
+            .fetch(&agent(), "9780306406157", None, CoverSize::Large)
+            .expect("an answer");
+        assert!(cover.is_some());
+    }
+
+    #[test]
+    fn covers_asked_by_isbn_are_spaced_out() {
+        let stub = Stub::start(|_, _| Reply::status(404));
+        let covers = Covers {
+            pace: Duration::from_millis(300),
+            ..stub.covers()
+        };
+        let started = Instant::now();
+        let found = covers.fetch(&agent(), "9780306406157", None, CoverSize::Large);
+        assert!(matches!(found, Ok(None)), "{found:?}");
+        // Two forms of the ISBN, so at least one pause between them.
+        assert!(started.elapsed() >= Duration::from_millis(300));
+        assert_eq!(stub.asked(), [ISBN13, ISBN10]);
+    }
+
     #[test]
     fn a_redirect_to_an_address_by_number_is_refused() {
         // A listener standing in for a service on the LAN; it only counts.

crates/bibfetch/src/lib.rs

@@ -112,6 +112,9 @@ pub enum Error {
     NoRegister(String),
     /// What was asked about is not an identifier of the kind it should be.
     Invalid(String),
+    /// The register refused because it has been asked too often from this
+    /// address. It is not down, and asking again later will work.
+    Limited(String),
 }
 
 impl std::fmt::Display for Error {
@@ -123,6 +126,7 @@ impl std::fmt::Display for Error {
                 write!(f, "no register here answers about {kind} identifiers")
             }
             Error::Invalid(why) => write!(f, "{why}"),
+            Error::Limited(why) => write!(f, "{why}"),
         }
     }
 }

crates/bibfetch/src/main.rs

@@ -225,6 +225,7 @@ fn cover(
             eprintln!("bibfetch: {isbn}: {why}");
             ExitCode::from(match why {
                 bibfetch::Error::Unreachable(_) => exit::UNAVAILABLE,
+                bibfetch::Error::Limited(_) => exit::TEMPFAIL,
                 _ => exit::DATA,
             })
         }

crates/bindings/python/catena/__init__.py

@@ -21,6 +21,8 @@ from ._catena import (
     compose_subjects,
     count_tokens,
     cue_windows,
+    Limited,
+    fetch_cover,
     fetch_identifier,
     find_citations,
     fuse_hits,
@@ -41,6 +43,7 @@ from ._catena import (
     record_severity,
     same_work,
     scrub,
+    search_title,
     segment_cues,
     series_of,
     shelve,
@@ -53,6 +56,7 @@ from ._catena import (
 
 __all__ = [
     "__version__",
+    "Limited",
     "canonical_subject",
     "chunk_markdown",
     "citation_prompt",
@@ -62,6 +66,7 @@ __all__ = [
     "compose_subjects",
     "count_tokens",
     "cue_windows",
+    "fetch_cover",
     "fetch_identifier",
     "find_citations",
     "fuse_hits",
@@ -82,6 +87,7 @@ __all__ = [
     "record_severity",
     "same_work",
     "scrub",
+    "search_title",
     "segment_cues",
     "series_of",
     "shelve",

crates/bindings/python/catena/__init__.pyi

@@ -103,6 +103,40 @@ def fetch_identifier(
     """
     ...
 
+class Limited(Exception):
+    """A register refused because it has been asked too often; ask again later."""
+
+def fetch_cover(
+    isbn: str,
+    cover_id: int | None = ...,
+    size: str = ...,
+    polite: str | None = ...,
+    timeout: int = ...,
+    deadline: int = ...,
+) -> dict[str, Any] | None:
+    """The cover Open Library has for a book, by ISBN, then by cover id.
+
+    A dict with `bytes`, `media_type`, `extension`, `source` and `url`, or
+    `None` when there is none. Raises `Limited` when Open Library refused for
+    being asked too often. Blocks on the network.
+    """
+    ...
+
+def search_title(
+    title: str,
+    author: str | None = ...,
+    limit: int = ...,
+    polite: str | None = ...,
+    timeout: int = ...,
+    attempts: int = ...,
+) -> list[dict[str, Any]]:
+    """The editions the registers list for a title and author, in their order.
+
+    Records shaped like `fetch_identifier`'s. Use `same_work` to decide which
+    one is the book. Blocks on the network.
+    """
+    ...
+
 def colophon_prompt(
     record: dict[str, Any], text: str, pages: int = ...
 ) -> str | None:

crates/bindings/src/python.rs

@@ -10,7 +10,7 @@
 
 use pyo3::exceptions::PyValueError;
 use pyo3::prelude::*;
-use pyo3::types::{PyDict, PyList};
+use pyo3::types::{PyBytes, PyDict, PyList};
 
 use crate::{ChunkBudgets, CueBudgets};
 
@@ -434,6 +434,9 @@ fn _catena(module: &Bound<'_, PyModule>) -> PyResult<()> {
     module.add_function(wrap_pyfunction!(read_colophon, module)?)?;
     module.add_function(wrap_pyfunction!(shelve, module)?)?;
     module.add_function(wrap_pyfunction!(fetch_identifier, module)?)?;
+    module.add_function(wrap_pyfunction!(fetch_cover, module)?)?;
+    module.add_function(wrap_pyfunction!(search_title, module)?)?;
+    module.add("Limited", module.py().get_type::<Limited>())?;
     module.add_function(wrap_pyfunction!(colophon_prompt, module)?)?;
     module.add_function(wrap_pyfunction!(merge_colophon, module)?)?;
     Ok(())
@@ -512,13 +515,112 @@ fn fetch_identifier<'py>(
     // would stop every other thread in the interpreter for the whole request.
     let found = py
         .detach(|| bibfetch::ask::lookup(&agent, id, kind, ask.attempts))
-        .map_err(|e| PyValueError::new_err(e.to_string()))?;
+        .map_err(register_error)?;
     match found {
         Some(record) => Ok(Some(to_py(py, &record)?)),
         None => Ok(None),
     }
 }
 
+pyo3::create_exception!(
+    _catena,
+    Limited,
+    pyo3::exceptions::PyException,
+    "A register refused because it has been asked too often; ask again later."
+);
+
+/// A register's error as the Python exception that says what to do about it.
+fn register_error(error: bibfetch::Error) -> PyErr {
+    match error {
+        bibfetch::Error::Limited(why) => Limited::new_err(why),
+        other => PyValueError::new_err(other.to_string()),
+    }
+}
+
+/// The cover Open Library has for a book, by its ISBN, then by the cover id
+/// a record named.
+///
+/// Returns a dict with `bytes`, `media_type` (`image/jpeg` or `image/png`),
+/// `extension`, `source` and `url`, or `None` when there is no cover. Covers
+/// asked by ISBN are kept three seconds apart across the process, and the
+/// whole fetch stops after `deadline` seconds. Raises `catena.Limited` when
+/// Open Library refused for being asked too often, and `ValueError` when the
+/// ISBN is not one or the store could not be reached. Blocks on the network.
+#[pyfunction]
+#[pyo3(signature = (isbn, cover_id = None, size = "large", polite = None, timeout = 20, deadline = 120))]
+fn fetch_cover<'py>(
+    py: Python<'py>,
+    isbn: &str,
+    cover_id: Option<u64>,
+    size: &str,
+    polite: Option<String>,
+    timeout: u64,
+    deadline: u64,
+) -> PyResult<Option<Bound<'py, PyDict>>> {
+    let size = match size {
+        "small" => bibfetch::CoverSize::Small,
+        "medium" => bibfetch::CoverSize::Medium,
+        "large" => bibfetch::CoverSize::Large,
+        other => {
+            return Err(PyValueError::new_err(format!(
+                "size is small, medium or large, not {other:?}"
+            )));
+        }
+    };
+    let agent = bibfetch::ask::Ask {
+        polite,
+        timeout: std::time::Duration::from_secs(timeout.max(1)),
+        ..Default::default()
+    }
+    .agent();
+    let covers = bibfetch::cover::Covers {
+        deadline: std::time::Duration::from_secs(deadline.max(1)),
+        ..Default::default()
+    };
+    let found = py
+        .detach(|| covers.fetch(&agent, isbn, cover_id, size))
+        .map_err(register_error)?;
+    let Some(cover) = found else {
+        return Ok(None);
+    };
+    let out = PyDict::new(py);
+    out.set_item("bytes", PyBytes::new(py, &cover.bytes))?;
+    out.set_item("media_type", cover.media_type)?;
+    out.set_item("extension", cover.extension())?;
+    out.set_item("source", cover.source)?;
+    out.set_item("url", cover.url)?;
+    Ok(Some(out))
+}
+
+/// The editions the registers list for a title, and an author if one is known.
+///
+/// Open Library is asked first and Google Books only when it has nothing.
+/// The candidates come back in the register's own order, as records shaped
+/// like `fetch_identifier`'s; nothing here decides which is the book, which is
+/// what `same_work` is for. Blocks on the network.
+#[pyfunction]
+#[pyo3(signature = (title, author = None, limit = 10, polite = None, timeout = 20, attempts = 3))]
+fn search_title<'py>(
+    py: Python<'py>,
+    title: &str,
+    author: Option<&str>,
+    limit: usize,
+    polite: Option<String>,
+    timeout: u64,
+    attempts: u32,
+) -> PyResult<Bound<'py, PyList>> {
+    let ask = bibfetch::ask::Ask {
+        polite,
+        timeout: std::time::Duration::from_secs(timeout.max(1)),
+        attempts: attempts.max(1),
+    };
+    let agent = ask.agent();
+    let found = py
+        .detach(|| bibfetch::search(&agent, title, author, limit, ask.attempts))
+        .map_err(register_error)?;
+    list_to_py(py, &found)
+}
+
 /// The question to put to a model about the fields a book's pages did not
 /// settle, or `None` when they settled everything.
 ///

crates/bookmeta/README.md

@@ -99,8 +99,10 @@ A Kindle book's title is EXTH 503 where it is there, then the MOBI header's
 full name, then the Palm database name (cut to 31 bytes, with a note). Authors
 (100) and subjects (105) are every record of their kind, in order; publisher
 (101), description (103), date (106) and language (524) the first. EXTH 104 is
-an `isbn` when it is shaped like one and `other` when not, and 113 is an
+an `isbn` when it is shaped like one and `other` when not. 113 is filed by
-`asin`. A contributor (108) is not an author, the same as an EPUB's
+its shape: a `uuid` when it is one (calibre writes its book id there), an
+`asin` when it is ten letters and digits, and `other` when it is neither. A
+contributor (108) is not an author, the same as an EPUB's
 `dc:contributor`, and is not reported. With no EXTH language, the language
 comes from the header's locale. Strings are decoded as the header says,
 UTF-8 or Windows-1252. A book whose text is encrypted still has its header
@@ -112,10 +114,55 @@ an EXTH block that breaks off keeps the records before the break.
 
 ## Covers
 
-`Metadata::cover` is the cover image a Kindle book carries (EXTH 201, or the
+`Metadata::cover` is the cover image the file carries, with its media type
-thumbnail, 202, when no cover is named), with its media type read off its
+read off its first bytes: JPEG, PNG, GIF, WebP or BMP. It is for library
-first bytes. It is for library callers and is not part of the JSON record,
+callers and is not part of the JSON record, which is a line of text.
-which is a line of text.
+
+A Kindle book's cover is EXTH 201, or the thumbnail, 202, when no cover is
+named. With neither, it is the record at the header's first image index, as
+calibre does, and only that record: when it is not a JPEG, PNG or GIF there is
+no cover, because the records after it can be illustrations, and kindlegen
+sometimes puts a font there first.
+
+An EPUB's is the first of these that reaches an image:
+
+1. The manifest item whose `properties` include `cover-image` (EPUB 3).
+2. The item `<meta name="cover" content="…">` names (EPUB 2). Some
+   converters write the image's address there rather than its id, and that is
+   followed too.
+3. The guide's `<reference type="cover">`, a page, and the `<img src>` or
+   SVG `<image href>` on it. Where the page has several, the one whose file
+   name, `alt`, `id` or `class` says `cover` is taken; where none does, the
+   first is, with a note that there were others. A guide that points straight
+   at an image is taken at its word.
+4. Any image in the manifest whose id or file name says `cover`, one named
+   `cover` outright first and the rest in manifest order.
+
+A name says `cover` when it has it as a word of its own, in any case. Words
+break at anything not a letter or digit and where a lower case letter meets a
+capital, so `CoverImage`, `Book_COVER` and `cover-thumb` say it, and
+`discover-ch1` and `recovery-map` do not.
+
+A route that names something unusable — not in the archive, not an image —
+leaves a note, and the next route is asked. Addresses are resolved against the
+document they are written in, with their fragment off and their
+percent-escapes undone, and one that climbs out of the archive or names a
+scheme (`http:`, `data:`) is not followed.
+
+What the bytes say beats what the manifest says: a JPEG listed as
+`image/png` comes back as `image/jpeg`, with a note. A cover larger than
+`bookmeta::MAX_COVER` (16 MiB) is left out with a note, in either format, and
+an EPUB's is never decompressed further than that, whatever size the archive
+claims for it. Nor are more than eight members read looking for one,
+however many candidates the manifest lists: the search stops there, with a
+note.
+
+`Cover::guessed` says the file left the choice open: a Kindle book's first
+image record, or an EPUB image picked because its name says `cover`. Either
+can be an illustration from inside the book, so a caller that can get a cover
+another way, from a register by ISBN, should prefer that one. A cover the file
+names is never a guess. `Cover::extension` gives the usual file extension for
+its media type.
 
 ## Asking a model
 

crates/bookmeta/src/lib.rs

@@ -106,10 +106,63 @@ pub struct Metadata {
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct Cover {
     /// What kind of image it is, read off its first bytes: `image/jpeg`,
-    /// `image/png`, `image/gif` or `image/bmp`.
+    /// `image/png`, `image/gif`, `image/webp` or `image/bmp`.
     pub media_type: String,
-    /// The image itself.
+    /// The image itself, never more than [`MAX_COVER`] bytes.
     pub bytes: Vec<u8>,
+    /// Whether the file left the choice open. A MOBI that names no cover, or
+    /// an EPUB whose only cover is an image with "cover" in its name, has one
+    /// picked by position or by name, and that pick can be an illustration
+    /// from inside the book. A caller with a better source should prefer it.
+    pub guessed: bool,
+}
+
+impl Cover {
+    /// The usual file extension for the image, without the dot.
+    pub fn extension(&self) -> &'static str {
+        match self.media_type.as_str() {
+            "image/png" => "png",
+            "image/gif" => "gif",
+            "image/webp" => "webp",
+            "image/bmp" => "bmp",
+            _ => "jpg",
+        }
+    }
+}
+
+/// How large a cover is worth keeping.
+///
+/// A real one is a few hundred kilobytes. An EPUB's cover has to be
+/// decompressed to be read, and this is as far as that goes: a larger one is
+/// left out with a note, in every format, so a hostile file cannot make a
+/// reader allocate whatever it likes.
+pub const MAX_COVER: usize = 16 * 1024 * 1024;
+
+/// A size, as a note says it.
+pub(crate) fn size(bytes: usize) -> String {
+    const MIB: usize = 1024 * 1024;
+    if bytes >= MIB && bytes % MIB == 0 {
+        format!("{} MiB", bytes / MIB)
+    } else {
+        format!("{bytes} bytes")
+    }
+}
+
+/// What kind of image these bytes are, by their magic.
+pub(crate) fn image_type(data: &[u8]) -> Option<&'static str> {
+    if data.starts_with(&[0xff, 0xd8, 0xff]) {
+        Some("image/jpeg")
+    } else if data.starts_with(b"\x89PNG\r\n\x1a\n") {
+        Some("image/png")
+    } else if data.starts_with(b"GIF87a") || data.starts_with(b"GIF89a") {
+        Some("image/gif")
+    } else if data.starts_with(b"RIFF") && data.get(8..12) == Some(b"WEBP") {
+        Some("image/webp")
+    } else if data.starts_with(b"BM") && data.len() > 14 {
+        Some("image/bmp")
+    } else {
+        None
+    }
 }
 
 fn version() -> u32 {

crates/bookmeta/src/mobi.rs

@@ -13,7 +13,7 @@
 //! panics or reads past the end.
 
 use crate::epub::{digits_of, is_isbn, is_uuid};
-use crate::{Cover, Metadata, Source};
+use crate::{Cover, MAX_COVER, Metadata, Source, image_type};
 
 /// Where a Palm database says what it holds.
 const TYPE_AT: usize = 60;
@@ -332,9 +332,17 @@ fn cover(
         meta.note("the cover record is not an image");
         return None;
     };
+    if data.len() > MAX_COVER {
+        meta.note(format!(
+            "the cover record is larger than {}",
+            crate::size(MAX_COVER)
+        ));
+        return None;
+    }
     Some(Cover {
         media_type: media_type.to_string(),
         bytes: data.to_vec(),
+        guessed: false,
     })
 }
 
@@ -347,10 +355,14 @@ fn cover(
 fn first_image_record(bytes: &[u8], records: &[usize], first_image: Option<u32>) -> Option<Cover> {
     let first = first_image.filter(|&index| index != NONE)?;
     let data = record(bytes, records, first as usize)?;
+    if data.len() > MAX_COVER {
+        return None;
+    }
     let media_type = image_type(data).filter(|kind| *kind != "image/bmp")?;
     Some(Cover {
         media_type: media_type.to_string(),
         bytes: data.to_vec(),
+        guessed: true,
     })
 }
 
@@ -359,21 +371,6 @@ fn is_asin(value: &str) -> bool {
     value.len() == 10 && value.chars().all(|c| c.is_ascii_alphanumeric())
 }
 
-/// What kind of image these bytes are, by their magic.
-fn image_type(data: &[u8]) -> Option<&'static str> {
-    if data.starts_with(&[0xff, 0xd8, 0xff]) {
-        Some("image/jpeg")
-    } else if data.starts_with(b"\x89PNG\r\n\x1a\n") {
-        Some("image/png")
-    } else if data.starts_with(b"GIF87a") || data.starts_with(b"GIF89a") {
-        Some("image/gif")
-    } else if data.starts_with(b"BM") && data.len() > 14 {
-        Some("image/bmp")
-    } else {
-        None
-    }
-}
-
 /// The Palm database's name, the last place a title can come from.
 ///
 /// Converters write the title there with spaces made underscores; it is

crates/bookmeta/tests/mobi.rs

@@ -31,6 +31,7 @@ fn says_what_it_was_given(meta: &bookmeta::Metadata) {
 
     let cover = meta.cover.as_ref().expect("a cover");
     assert_eq!(cover.media_type, "image/jpeg");
+    assert!(!cover.guessed, "EXTH 201 names the cover");
     let original = std::fs::read(fixture("lantern-cover.jpg")).expect("the cover fixture");
     assert_eq!(cover.bytes, original);
 }
@@ -84,6 +85,7 @@ fn a_file_naming_no_cover_gets_its_first_image_record() {
     let meta = bookmeta::from_bytes("x.mobi", &lantern_naming_no_cover());
     let cover = meta.cover.expect("the first image record");
     assert_eq!(cover.media_type, "image/jpeg");
+    assert!(cover.guessed, "a cover taken by position is a guess");
     let original = std::fs::read(fixture("lantern-cover.jpg")).expect("the cover fixture");
     assert_eq!(cover.bytes, original);
 }

crates/booksniff/README.md

@@ -26,8 +26,8 @@ file too small to be a book, a name that says "sample".
   cover images that ended up in a folder of PDFs.
 
 A library accumulates files that lie: a DjVu named `.pdf`, a Word document
-named `.epub`, an EPUB saved as `.zip`, a single PDF that somebody zipped and
+named `.epub`, an RTF saved as `.doc`, an EPUB saved as `.zip`, a single PDF
-renamed back to `.pdf`.
+that somebody zipped and renamed back to `.pdf`.
 
 ## Install
 
@@ -89,7 +89,12 @@ one bad file does not stop a sweep.
 | `ole2` | The compound-file header every binary Word document starts with |
 | `epub` | A zip holding `mimetype` and `META-INF/container.xml` |
 | `zip-wrapped-pdf` | A zip whose only real member is one PDF |
+| `zip-wrapped-fb2` | A zip whose only real member is one FictionBook — the `.fb2.zip` most of them are passed around as |
+| `cbz` | A zip whose every real member is a page image: JPEG, PNG, GIF or WebP |
 | `azw3`, `mobi` | A Palm database tagged `BOOKMOBI`, told apart by the format version inside the first record |
+| `lit` | `ITOLITLS`, Microsoft Reader's signature |
+| `rtf` | `{\rtf` at the front, after any byte-order mark and blank space |
+| `fb2` | XML whose root element is `FictionBook`, found by stepping over the declaration, comments and doctype in the first four kilobytes |
 | `html` | Markup at the front, after any byte-order mark |
 | `txt` | Text that is none of the above |
 | `unknown` | Anything else, with a note saying what was seen |
@@ -98,6 +103,20 @@ An archive holding several PDFs is a collection, not a wrapped book: choosing
 among them would be a guess, and naming one chapter as the whole work is worse
 than refusing.
 
+A zipped FictionBook is reported as wrapped rather than as `fb2`, the same way
+a zipped PDF is: a converter handed it as FictionBook is handed a zip, and the
+kind is there to stop exactly that.
+
+A comic is the pages and nothing else. Directories, the `ComicInfo.xml` a
+comic reader writes into the archive, and the desktop clutter every archive
+picks up are passed over; anything more — one PDF among the pages, a readme —
+and the zip is not a comic, and says what it holds instead. Only names are
+read, out of the archive's directory, so no member is ever unpacked.
+
+FictionBook is read as bytes rather than as text. A great many are written in
+windows-1251, which spells its markup in ASCII but is not UTF-8, and would
+otherwise be called binary.
+
 ## As a library
 
 ```rust

crates/booksniff/src/fb2.rs

@@ -0,0 +1,162 @@
+//! Telling a FictionBook from any other XML.
+//!
+//! A FictionBook is an XML document whose root element is `FictionBook`.
+//! The root comes after the prolog — a declaration, comments, perhaps a
+//! doctype — so it is found by stepping over those rather than by matching
+//! at a fixed offset.
+//!
+//! The walk is over bytes, not text. A great many FictionBooks are written in
+//! windows-1251, which is not UTF-8 but spells its markup in ASCII all the
+//! same, and a reader that insisted on UTF-8 would call every one of them
+//! binary.
+
+use crate::BOM;
+
+/// What the root element of a FictionBook is called.
+const ROOT: &[u8] = b"FictionBook";
+
+/// Whether the front of a file is the start of a FictionBook.
+///
+/// `head` is however much of the front the caller read. A root element that
+/// does not begin inside it is not found, so the answer is no rather than a
+/// guess.
+pub fn is_fictionbook(head: &[u8]) -> bool {
+    root_element(head).is_some_and(|name| local_name(name) == ROOT)
+}
+
+/// The name of the first element, after the prolog.
+fn root_element(head: &[u8]) -> Option<&[u8]> {
+    let mut rest = head.strip_prefix(BOM).unwrap_or(head);
+    loop {
+        rest = rest.trim_ascii_start();
+        if let Some(after) = rest.strip_prefix(b"<?") {
+            rest = after_marker(after, b"?>")?;
+        } else if let Some(after) = rest.strip_prefix(b"<!--") {
+            rest = after_marker(after, b"-->")?;
+        } else if let Some(after) = rest.strip_prefix(b"<!") {
+            rest = after_declaration(after)?;
+        } else {
+            // Anything but an element here is text before the root, which
+            // XML does not allow.
+            let after = rest.strip_prefix(b"<")?;
+            // A name that runs to the end of what was read may be the front
+            // of a longer one, so it is not taken.
+            let end = after.iter().position(|byte| !is_name_byte(*byte))?;
+            let ends_properly =
+                matches!(after[end], b'>' | b'/') || after[end].is_ascii_whitespace();
+            return (end > 0 && ends_properly).then_some(&after[..end]);
+        }
+    }
+}
+
+/// What follows the first `marker`, if it is there at all.
+fn after_marker<'a>(rest: &'a [u8], marker: &[u8]) -> Option<&'a [u8]> {
+    rest.windows(marker.len())
+        .position(|window| window == marker)
+        .map(|at| &rest[at + marker.len()..])
+}
+
+/// What follows a doctype, whose internal subset may hold `>` of its own.
+fn after_declaration(rest: &[u8]) -> Option<&[u8]> {
+    let mut depth = 0usize;
+    let mut quote = None;
+    for (at, &byte) in rest.iter().enumerate() {
+        match quote {
+            Some(open) if byte == open => quote = None,
+            Some(_) => {}
+            None => match byte {
+                b'"' | b'\'' => quote = Some(byte),
+                b'[' => depth += 1,
+                b']' => depth = depth.saturating_sub(1),
+                b'>' if depth == 0 => return Some(&rest[at + 1..]),
+                _ => {}
+            },
+        }
+    }
+    None
+}
+
+/// The name without its namespace prefix.
+fn local_name(name: &[u8]) -> &[u8] {
+    match name.iter().rposition(|byte| *byte == b':') {
+        Some(at) => &name[at + 1..],
+        None => name,
+    }
+}
+
+fn is_name_byte(byte: u8) -> bool {
+    byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-' | b'.' | b':') || byte >= 0x80
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    const BOOK: &[u8] = b"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n\
+        <FictionBook xmlns=\"http://www.gribuser.ru/xml/fictionbook/2.0\">\n\
+        <description><title-info><book-title>The Mill</book-title></title-info></description>";
+
+    #[test]
+    fn a_fictionbook_is_recognised() {
+        assert!(is_fictionbook(BOOK));
+        assert!(is_fictionbook(b"<FictionBook>"));
+    }
+
+    #[test]
+    fn the_prolog_is_stepped_over() {
+        let head = b"\xef\xbb\xbf  <?xml version=\"1.0\"?>\r\n\
+            <!-- converted by hand -->\n\
+            <!DOCTYPE FictionBook [ <!ENTITY mill \"<b>the mill</b>\"> ]>\n\
+            <?xml-stylesheet href=\"fb2.css\"?>\n\
+            <FictionBook\n xmlns=\"http://www.gribuser.ru/xml/fictionbook/2.0\">";
+        assert!(is_fictionbook(head));
+    }
+
+    #[test]
+    fn a_prefixed_root_is_still_a_fictionbook() {
+        assert!(is_fictionbook(
+            b"<fb:FictionBook xmlns:fb=\"http://www.gribuser.ru/xml/fictionbook/2.0\">"
+        ));
+    }
+
+    #[test]
+    fn markup_need_not_be_utf8() {
+        // windows-1251 for "Мельница" in the title.
+        let head = b"<?xml version=\"1.0\" encoding=\"windows-1251\"?>\
+            <FictionBook><description><book-title>\xcc\xe5\xeb\xfc\xed\xe8\xf6\xe0</book-title>";
+        assert!(is_fictionbook(head));
+    }
+
+    #[test]
+    fn other_xml_is_not_a_fictionbook() {
+        assert!(!is_fictionbook(
+            b"<?xml version=\"1.0\"?>\n<rss version=\"2.0\">"
+        ));
+        assert!(!is_fictionbook(b"<?xml version=\"1.0\"?><FictionBookish>"));
+        assert!(!is_fictionbook(b"<html><FictionBook>"));
+        assert!(!is_fictionbook(b"text before <FictionBook>"));
+        assert!(!is_fictionbook(b""));
+    }
+
+    #[test]
+    fn a_root_the_window_cut_short_is_not_taken() {
+        assert!(!is_fictionbook(b"<?xml version=\"1.0\"?><FictionBook"));
+        assert!(!is_fictionbook(b"<?xml version=\"1.0\"?><Fiction"));
+        assert!(!is_fictionbook(b"<?xml version=\"1.0\""));
+        assert!(!is_fictionbook(b"<!-- never closed <FictionBook>"));
+    }
+
+    #[test]
+    fn a_truncated_or_mangled_prolog_does_not_panic() {
+        let head = b"\xef\xbb\xbf<?xml?><!--c--><!DOCTYPE x [ <!ENTITY a '>'> ] \"]\">\
+            <FictionBook>";
+        assert!(is_fictionbook(head));
+        for cut in 0..head.len() {
+            let _ = is_fictionbook(&head[..cut]);
+        }
+        for byte in 0..=255u8 {
+            let _ = is_fictionbook(&[b'<', byte]);
+            let _ = is_fictionbook(&[b'<', b'!', byte, b'>']);
+        }
+    }
+}

crates/booksniff/src/lib.rs

@@ -9,6 +9,7 @@
 //! The containers are unambiguous, so they are read rather than guessed at.
 
 pub mod bytes;
+pub mod fb2;
 pub mod mobi;
 pub mod zip;
 
@@ -22,17 +23,32 @@ const DJVU: &[u8] = b"AT&TFORM";
 const OLE2: &[u8] = &[0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1];
 /// A PDF announces its version in its first bytes.
 const PDF: &[u8] = b"%PDF-";
+/// Microsoft Reader's signature, ahead of everything else in the file.
+const LIT: &[u8] = b"ITOLITLS";
+/// The group every RTF document is, opened with the control word that names
+/// the format.
+const RTF: &[u8] = br"{\rtf";
+/// The UTF-8 byte-order mark.
+const BOM: &[u8] = b"\xef\xbb\xbf";
 
 /// Archive members that carry no content: resource forks and desktop clutter.
 const ARCHIVE_NOISE: &[&str] = &["__macosx/", ".ds_store", "thumbs.db"];
 
+/// What a comic book archive carries besides its pages: the catalogue entry
+/// comic readers write into it.
+const COMIC_NOISE: &[&str] = &["comicinfo.xml"];
+
+/// The extensions of the images a comic book archive is made of.
+const PAGE_IMAGES: &[&str] = &[".jpg", ".jpeg", ".png", ".gif", ".webp"];
+
 /// Substrings in a name that mark a file as a sample rather than a book.
 const SAMPLE_MARKERS: &[&str] = &["sample", "readme", "read me"];
 
 /// Below this, a file is too small to be a book whatever it claims to be.
 const SMALLEST_BOOK: u64 = 4096;
 
-/// How much of the front is read to tell text from markup.
+/// How much of the front is read to tell RTF, FictionBook, markup and text
+/// apart.
 const TEXT_WINDOW: usize = 4096;
 
 /// Identify a file.
@@ -49,12 +65,17 @@ pub fn sniff(source: &impl Bytes, name: &str) -> Doc {
         Kind::Pdf
     } else if source.magic_at(0, OLE2) {
         Kind::Ole2
+    } else if source.magic_at(0, LIT) {
+        Kind::Lit
     } else if source.magic_at(0, zip::LOCAL_HEADER) {
         archive(source, &mut doc)
     } else if let Some(kind) = mobi::kind(source) {
         kind
     } else {
-        text(source, &mut doc)
+        match source.slice(0, TEXT_WINDOW) {
+            Ok(head) => front(&head, &mut doc),
+            Err(_) => Kind::Unknown,
+        }
     };
 
     if doc.kind == Kind::Unknown {
@@ -80,7 +101,8 @@ pub fn sniff(source: &impl Bytes, name: &str) -> Doc {
     doc
 }
 
-/// What is inside a zip: an EPUB, a wrapped PDF, or a collection of pages.
+/// What is inside a zip: an EPUB, a wrapped book, a comic, or a collection of
+/// pages.
 fn archive(source: &impl Bytes, doc: &mut Doc) -> Kind {
     let Some(names) = zip::members(source) else {
         doc.note("a zip whose directory could not be read");
@@ -99,6 +121,13 @@ fn archive(source: &impl Bytes, doc: &mut Doc) -> Kind {
         doc.note(format!("a zip holding one PDF, {pdf:?}"));
         return Kind::ZipWrappedPdf;
     }
+    if let Some(fb2) = sole_member(&names, ".fb2") {
+        doc.note(format!("a zip holding one FictionBook, {fb2:?}"));
+        return Kind::ZipWrappedFb2;
+    }
+    if is_comic(&names) {
+        return Kind::Cbz;
+    }
     if lower
         .iter()
         .any(|name| name.ends_with(".htm") || name.ends_with(".html") || name.ends_with(".xhtml"))
@@ -119,25 +148,67 @@ fn archive(source: &impl Bytes, doc: &mut Doc) -> Kind {
 /// a collection, and choosing among them would be a guess — better to say it
 /// is not a book than to name one chapter as the whole.
 pub fn sole_pdf(names: &[String]) -> Option<&str> {
-    let real: Vec<&String> = names
+    sole_member(names, ".pdf")
+}
+
+/// The one member an archive exists to carry, if it is the only real member
+/// and its name ends in `extension`.
+fn sole_member<'a>(names: &'a [String], extension: &str) -> Option<&'a str> {
+    match real_members(names).as_slice() {
+        [only] if only.to_lowercase().ends_with(extension) => Some(only.as_str()),
+        _ => None,
+    }
+}
+
+/// Whether an archive is the pages of a comic and nothing else.
+///
+/// Every real member has to be an image. One PDF or one text file among the
+/// pages makes it something else, and saying what would be a guess.
+fn is_comic(names: &[String]) -> bool {
+    let pages: Vec<String> = real_members(names)
+        .into_iter()
+        .map(|name| name.to_lowercase())
+        .filter(|name| {
+            let file = name.rsplit('/').next().unwrap_or(name);
+            !COMIC_NOISE.contains(&file)
+        })
+        .collect();
+    !pages.is_empty()
+        && pages
+            .iter()
+            .all(|name| PAGE_IMAGES.iter().any(|image| name.ends_with(image)))
+}
+
+/// The members that carry content: no directories, no desktop clutter.
+fn real_members(names: &[String]) -> Vec<&String> {
+    names
         .iter()
         .filter(|name| !name.ends_with('/'))
         .filter(|name| {
             let lower = name.to_lowercase();
             !ARCHIVE_NOISE.iter().any(|noise| lower.contains(noise))
         })
-        .collect();
+        .collect()
-    match real.as_slice() {
+}
-        [only] if only.to_lowercase().ends_with(".pdf") => Some(only.as_str()),
+
-        _ => None,
+/// What the front of a file says when it is not a container: RTF, a
+/// FictionBook, markup, text, or nothing this knows.
+fn front(head: &[u8], doc: &mut Doc) -> Kind {
+    // Both are recognised on bytes rather than text. RTF is seven-bit by
+    // design but not always in practice, and a FictionBook is as often
+    // windows-1251 as UTF-8.
+    let lead = head.strip_prefix(BOM).unwrap_or(head).trim_ascii_start();
+    if lead.starts_with(RTF) {
+        return Kind::Rtf;
     }
+    if fb2::is_fictionbook(head) {
+        return Kind::Fb2;
+    }
+    text(head, doc)
 }
 
 /// Text, markup, or nothing this knows.
-fn text(source: &impl Bytes, doc: &mut Doc) -> Kind {
+fn text(head: &[u8], doc: &mut Doc) -> Kind {
-    let Ok(head) = source.slice(0, TEXT_WINDOW) else {
-        return Kind::Unknown;
-    };
     if head.is_empty() {
         doc.note("the file is empty");
         return Kind::Unknown;
@@ -149,7 +220,7 @@ fn text(source: &impl Bytes, doc: &mut Doc) -> Kind {
     }
     // The window may have cut a character in half, which is not the same
     // thing as the file not being text.
-    let head = match std::str::from_utf8(&head) {
+    let head = match std::str::from_utf8(head) {
         Ok(text) => text,
         Err(e) if e.error_len().is_none() => {
             std::str::from_utf8(&head[..e.valid_up_to()]).expect("a valid prefix")
@@ -178,10 +249,15 @@ fn names_the_same_thing(extension: &str, kind: Kind) -> bool {
         Kind::Djvu => matches!(extension, "djvu" | "djv"),
         Kind::Azw3 => matches!(extension, "azw3" | "azw" | "kfx"),
         Kind::Mobi => matches!(extension, "mobi" | "prc" | "azw"),
+        Kind::Lit => extension == "lit",
+        Kind::Fb2 => extension == "fb2",
+        Kind::Rtf => extension == "rtf",
         Kind::Txt => matches!(extension, "txt" | "text" | "md" | "markdown"),
         Kind::Html => matches!(extension, "html" | "htm" | "xhtml" | "xml"),
         Kind::Ole2 => matches!(extension, "doc" | "xls" | "ppt"),
         Kind::ZipWrappedPdf => extension == "zip",
+        Kind::ZipWrappedFb2 => matches!(extension, "zip" | "fbz"),
+        Kind::Cbz => extension == "cbz",
         // Nothing was recognised, so there is nothing to disagree with.
         Kind::Unknown => true,
     }
@@ -194,10 +270,15 @@ fn spelling(kind: Kind) -> &'static str {
         Kind::Djvu => "DjVu",
         Kind::Azw3 => "AZW3",
         Kind::Mobi => "MOBI",
+        Kind::Lit => "Microsoft Reader",
+        Kind::Fb2 => "FictionBook",
+        Kind::Rtf => "RTF",
         Kind::Txt => "plain text",
         Kind::Html => "HTML",
         Kind::Ole2 => "an OLE2 document",
         Kind::ZipWrappedPdf => "a zip holding one PDF",
+        Kind::ZipWrappedFb2 => "a zip holding one FictionBook",
+        Kind::Cbz => "a comic book archive",
         Kind::Unknown => "nothing recognisable",
     }
 }
@@ -332,4 +413,164 @@ mod tests {
         head.truncate(TEXT_WINDOW + 1);
         assert_eq!(sniff(&head.as_slice(), "a.txt").kind, Kind::Txt);
     }
+
+    fn archive_of(names: &[&str], name: &str) -> Doc {
+        sniff(&zip::tests::zip(names).as_slice(), name)
+    }
+
+    #[test]
+    fn rtf_is_recognised_by_its_opening_group() {
+        assert_eq!(kind_of(br"{\rtf1\ansi\deff0 {\fonttbl", "a.rtf"), Kind::Rtf);
+        assert_eq!(
+            kind_of(b"\xef\xbb\xbf\r\n  {\\rtf1\\ansi", "a.rtf"),
+            Kind::Rtf
+        );
+        // Eight-bit text in the body does not stop it being RTF.
+        assert_eq!(kind_of(b"{\\rtf1 caf\xe9 \x00", "a.rtf"), Kind::Rtf);
+    }
+
+    #[test]
+    fn rtf_cut_short_of_its_control_word_is_not_rtf() {
+        assert_eq!(sniff(&br"{\rt".as_slice(), "a.rtf").kind, Kind::Txt);
+        assert_eq!(kind_of(br"{\rtx1", "a.rtf"), Kind::Txt);
+        assert_eq!(kind_of(br"text {\rtf1", "a.rtf"), Kind::Txt);
+    }
+
+    #[test]
+    fn rtf_named_as_a_word_document_is_remarked_on() {
+        let doc = sniff(&padded(br"{\rtf1\ansi").as_slice(), "letter.doc");
+        assert_eq!(doc.kind, Kind::Rtf);
+        assert!(
+            doc.notes
+                .iter()
+                .any(|note| note == "named .doc, but the bytes say RTF"),
+            "{:?}",
+            doc.notes
+        );
+    }
+
+    #[test]
+    fn microsoft_reader_is_recognised_by_its_signature() {
+        let doc = sniff(&padded(b"ITOLITLS\x01\x00\x00\x00").as_slice(), "book.lit");
+        assert_eq!(doc.kind, Kind::Lit);
+        assert!(doc.notes.is_empty(), "{:?}", doc.notes);
+        assert_eq!(kind_of(b"ITOLITL", "book.lit"), Kind::Txt);
+        assert_eq!(kind_of(b" ITOLITLS", "book.lit"), Kind::Txt);
+    }
+
+    #[test]
+    fn a_fictionbook_is_recognised_and_other_xml_is_not() {
+        let book = b"<?xml version=\"1.0\" encoding=\"windows-1251\"?>\n\
+            <FictionBook xmlns=\"http://www.gribuser.ru/xml/fictionbook/2.0\">\
+            <book-title>\xcc\xe5\xeb\xfc\xed\xe8\xf6\xe0</book-title>";
+        let doc = sniff(&padded(book).as_slice(), "mill.fb2");
+        assert_eq!(doc.kind, Kind::Fb2);
+        assert!(doc.notes.is_empty(), "{:?}", doc.notes);
+
+        let feed = b"<?xml version=\"1.0\"?>\n<rss version=\"2.0\"><channel>";
+        assert_eq!(kind_of(feed, "feed.xml"), Kind::Html);
+    }
+
+    #[test]
+    fn a_fictionbook_named_as_something_else_is_remarked_on() {
+        let doc = sniff(&padded(b"<FictionBook>").as_slice(), "mill.txt");
+        assert_eq!(doc.kind, Kind::Fb2);
+        assert!(
+            doc.notes.iter().any(|note| note.contains("FictionBook")),
+            "{:?}",
+            doc.notes
+        );
+    }
+
+    #[test]
+    fn a_zip_holding_one_fictionbook_is_a_wrapped_one() {
+        let doc = archive_of(&["__MACOSX/._mill.fb2", "mill.fb2"], "mill.fb2.zip");
+        assert_eq!(doc.kind, Kind::ZipWrappedFb2);
+        assert_eq!(doc.declared_ext.as_deref(), Some("zip"));
+        assert!(
+            doc.notes
+                .iter()
+                .any(|note| note.contains("one FictionBook")),
+            "{:?}",
+            doc.notes
+        );
+        assert!(!doc.notes.iter().any(|note| note.contains("named .")));
+
+        let two = archive_of(&["one.fb2", "two.fb2"], "pair.zip");
+        assert_eq!(two.kind, Kind::Unknown);
+    }
+
+    #[test]
+    fn a_zip_of_page_images_is_a_comic() {
+        let doc = archive_of(
+            &[
+                "Issue 1/",
+                "Issue 1/001.jpg",
+                "Issue 1/002.JPEG",
+                "Issue 1/003.png",
+                "Issue 1/004.gif",
+                "Issue 1/005.webp",
+                "ComicInfo.xml",
+                "__MACOSX/Issue 1/._001.jpg",
+                "Issue 1/Thumbs.db",
+                ".DS_Store",
+            ],
+            "issue-1.cbz",
+        );
+        assert_eq!(doc.kind, Kind::Cbz);
+        assert!(
+            !doc.notes.iter().any(|note| note.contains("named .")),
+            "{:?}",
+            doc.notes
+        );
+    }
+
+    #[test]
+    fn a_comic_saved_as_a_zip_is_remarked_on() {
+        let doc = archive_of(&["001.jpg", "002.jpg"], "issue-1.zip");
+        assert_eq!(doc.kind, Kind::Cbz);
+        assert!(
+            doc.notes.iter().any(|note| note.contains("comic")),
+            "{:?}",
+            doc.notes
+        );
+    }
+
+    #[test]
+    fn a_zip_with_anything_but_images_is_not_a_comic() {
+        let one_pdf = archive_of(&["001.jpg", "002.jpg", "extras.pdf"], "issue-1.cbz");
+        assert_eq!(one_pdf.kind, Kind::Unknown);
+        assert!(
+            one_pdf
+                .notes
+                .iter()
+                .any(|note| note.contains("none of them a book")),
+            "{:?}",
+            one_pdf.notes
+        );
+
+        let readme = archive_of(&["001.jpg", "readme.txt"], "issue-1.cbz");
+        assert_eq!(readme.kind, Kind::Unknown);
+
+        // A name that ends like an image without being one.
+        let lookalike = archive_of(&["001.jpg", "notes.jpg.txt"], "issue-1.cbz");
+        assert_eq!(lookalike.kind, Kind::Unknown);
+    }
+
+    #[test]
+    fn a_zip_with_no_pages_is_not_a_comic() {
+        let catalogue_only = archive_of(&["ComicInfo.xml", "pages/"], "issue-1.cbz");
+        assert_eq!(catalogue_only.kind, Kind::Unknown);
+
+        let empty = archive_of(&[], "issue-1.cbz");
+        assert_eq!(empty.kind, Kind::Unknown);
+    }
+
+    #[test]
+    fn a_truncated_comic_does_not_panic() {
+        let whole = zip::tests::zip(&["001.jpg", "002.png", "ComicInfo.xml"]);
+        for cut in 0..whole.len() {
+            let _ = sniff(&&whole[..cut], "issue-1.cbz");
+        }
+    }
 }

crates/booksniff/src/zip.rs

@@ -78,11 +78,11 @@ fn directory_start(source: &impl Bytes) -> Option<(usize, u64)> {
 }
 
 #[cfg(test)]
-mod tests {
+pub(crate) mod tests {
     use super::*;
 
     /// Build a zip holding `names`, each with empty stored content.
-    fn zip(names: &[&str]) -> Vec<u8> {
+    pub(crate) fn zip(names: &[&str]) -> Vec<u8> {
         let mut out: Vec<u8> = Vec::new();
         let mut offsets = Vec::new();
         for name in names {

crates/booksniff/tests/cli.rs

@@ -64,6 +64,30 @@ fn several_files_come_back_as_several_records() {
     assert_eq!(kinds, ["\"pdf\"", "\"djvu\""]);
 }
 
+#[test]
+fn the_formats_read_from_the_front_come_back_by_their_wire_names() {
+    let rtf = written("letter.rtf", &padded(br"{\rtf1\ansi"));
+    let lit = written("book.lit", &padded(b"ITOLITLS"));
+    let fb2 = written(
+        "mill.fb2",
+        &padded(b"<?xml version=\"1.0\"?>\n<FictionBook xmlns=\"http://www.gribuser.ru/xml/fictionbook/2.0\">"),
+    );
+    let out = run(&[
+        rtf.to_str().expect("utf-8 path"),
+        lit.to_str().expect("utf-8 path"),
+        fb2.to_str().expect("utf-8 path"),
+    ]);
+    assert_eq!(out.status.code(), Some(0));
+    let kinds: Vec<String> = String::from_utf8(out.stdout)
+        .expect("utf-8")
+        .lines()
+        .map(|line| {
+            serde_json::from_str::<serde_json::Value>(line).expect("valid JSON")["kind"].to_string()
+        })
+        .collect();
+    assert_eq!(kinds, ["\"rtf\"", "\"lit\"", "\"fb2\""]);
+}
+
 #[test]
 fn a_file_that_is_not_there_is_an_io_error_and_the_rest_still_run() {
     let path = written("kept.pdf", &padded(b"%PDF-1.7"));

crates/corpus-core/src/doc.rs

@@ -77,6 +77,12 @@ pub enum Kind {
     Azw3,
     /// Amazon MOBI.
     Mobi,
+    /// Microsoft Reader.
+    Lit,
+    /// FictionBook, the XML ebook format.
+    Fb2,
+    /// Rich Text Format.
+    Rtf,
     /// Plain text.
     Txt,
     /// HTML or XHTML.
@@ -85,6 +91,10 @@ pub enum Kind {
     Ole2,
     /// A zip archive whose only real content is a single PDF.
     ZipWrappedPdf,
+    /// A zip archive whose only real content is a single FictionBook.
+    ZipWrappedFb2,
+    /// A comic book archive: a zip of page images.
+    Cbz,
     /// Recognised as none of the above.
     Unknown,
 }
@@ -118,6 +128,11 @@ mod tests {
             (Kind::Ole2, "\"ole2\""),
             (Kind::Azw3, "\"azw3\""),
             (Kind::ZipWrappedPdf, "\"zip-wrapped-pdf\""),
+            (Kind::ZipWrappedFb2, "\"zip-wrapped-fb2\""),
+            (Kind::Fb2, "\"fb2\""),
+            (Kind::Lit, "\"lit\""),
+            (Kind::Rtf, "\"rtf\""),
+            (Kind::Cbz, "\"cbz\""),
             (Kind::Unknown, "\"unknown\""),
         ];
         for (kind, wire) in cases {

crates/corpus-core/src/exit.rs

@@ -30,6 +30,8 @@ pub const UNAVAILABLE: u8 = 69;
 pub const SOFTWARE: u8 = 70;
 /// Reading or writing failed.
 pub const IO: u8 = 74;
+/// A service refused for now, and asking again later will work.
+pub const TEMPFAIL: u8 = 75;
 
 /// The answer a predicate tool gives.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]

docs/contract.md

@@ -47,6 +47,7 @@ that was concatenated or written on Windows still reads.
 | `69` | A program it needs is not installed |
 | `70` | Its own fault |
 | `74` | Reading or writing failed |
+| `75` | A service refused for now; try again later |
 
 Failures start where `sysexits.h` puts them, so "no" and "broke" are never the
 same number. That is what makes this work:

docs/decisions.md

@@ -20,6 +20,7 @@ to any of them is a change to a decision rather than to an accident.
 - [pdfgutter — a page tree that does not add up is refused](#pdfgutter--a-page-tree-that-does-not-add-up-is-refused)
 - [pdfgutter — a sheet on its side is cut the way it is read](#pdfgutter--a-sheet-on-its-side-is-cut-the-way-it-is-read)
 - [pdfgutter — an unreadable file is an answer, not a false](#pdfgutter--an-unreadable-file-is-an-answer-not-a-false)
+- [bookmeta — a cover is what its bytes say it is](#bookmeta--a-cover-is-what-its-bytes-say-it-is)
 - [Every pattern — two regex engines, two Unicode tables](#every-pattern--two-regex-engines-two-unicode-tables)
 
 ## cuenorm — settings after the end time are the format, not a malformation
@@ -243,6 +244,23 @@ answers, and the [contract](contract.md) gives them three different numbers. A
 caller wanting them collapsed treats 1 and 2 alike; a caller wanting to know
 which files it failed on can now find out.
 
+## bookmeta — a cover is what its bytes say it is
+
+An EPUB's manifest gives every image a media type, and the cover's is not
+used: the type comes from the image's first bytes, and a disagreement is a
+note. Elsewhere `bookmeta` reports what the file says rather than what is
+true, and this is the one place it does not.
+
+The difference is that a title nobody believes is still a title, while a
+media type that is wrong is an image that does not open. A caller handed
+`image/png` for a JPEG writes `cover.png`, serves it as PNG, or hands it to a
+decoder that refuses it. An image can be replaced by a converter or by hand
+without its manifest entry changing, and the note keeps what the file said,
+so nothing is lost by correcting it.
+
+The same reasoning goes the other way: bytes that are not an image are not a
+cover, whatever the manifest calls them.
+
 ## Every pattern — two regex engines, two Unicode tables
 
 Patterns here spell their character classes explicitly rather than leaning on

docs/records.md

@@ -19,7 +19,7 @@ Part of [catena](../README.md).
 | Field | Meaning |
 |---|---|
 | `path` | Where the file was read from, as given |
-| `kind` | `pdf`, `epub`, `djvu`, `azw3`, `mobi`, `txt`, `html`, `ole2`, `zip-wrapped-pdf`, `unknown` |
+| `kind` | `pdf`, `epub`, `djvu`, `azw3`, `mobi`, `lit`, `fb2`, `rtf`, `txt`, `html`, `ole2`, `cbz`, `zip-wrapped-pdf`, `zip-wrapped-fb2`, `unknown` |
 | `declared_ext` | The extension the name claimed, lowercased, without the dot |
 | `bytes` | Size on disk |
 | `pages` | Page count, for paginated formats |