catena/crates/bookmeta/README.md

bookmeta

The metadata a file already carries.

Part of catena.

What it does

Every ebook format has somewhere to put the title, and almost every file in a library has something written there — by the publisher, by the converter, or by whoever last touched it. Reading it costs nothing, needs no network, and is the first thing a catalogue wants.

What comes back is what the file says, not what is true. A PDF titled Microsoft Word - ch3.doc is reported with that title and a note saying where it came from. Deciding that is rubbish is bibclean's job; finding out what is actually true is bibfetch's.

When you would use it

• Cataloguing a directory you have just been given, before anything else.
• Filling a library without shelling out to calibre for every file.
• Finding the books that say nothing about themselves, which are the ones worth a person's attention.

Install

cargo install --git https://git.pumpin.online/funkpower/catena bookmeta

Example

$ bookmeta library/*.epub | jq -r '[.title, (.authors|join(", "))] | @tsv'
Hagakure - The Way of the Samurai	Yamamoto Tsunetomo
The Complete Stories	Flannery O'Connor
A Collection of Essays	George Orwell

The whole record:

$ bookmeta philokalia.epub
{"v":1,"path":"philokalia.epub","source":"opf","title":"The Philokalia, Volume 3","authors":["G. E. H. Palmer","Kallistos Ware"],"publisher":"Faber & Faber","date":"1984-09-17","language":"en","series":"The Philokalia","series_index":"3","identifiers":{"isbn":"9780571175256"},"subjects":["Orthodox Christianity"]}

The ones that said nothing:

bookmeta library/* | jq -r 'select(.title == null) | .path'

Feeding a catalogue:

bookmeta library/*.epub \
  | jq -c '{title, author: .authors[0], isbn: .identifiers.isbn}' \
  | bibclean

Options

Option	Effect
--prompt	Write a question about what the file did not say, instead of the record
--merge	Read a model's reply on stdin and fill in what was missing
--excerpt <FILE>	Text to show the model alongside the metadata
--man	Write this page as a manual page, in roff

Exit codes

Code	Meaning
0	Every file said something
1	At least one said nothing
64	The command line was wrong
65	A model's reply could not be read
74	A file could not be read

Where it reads from

Format	Where
EPUB	The package document named by META-INF/container.xml — Dublin Core, plus the <meta> entries Calibre adds for a series
PDF	The trailer's information dictionary
MOBI, AZW, AZW3	The MOBI header in the first record, and the EXTH records after it

An EPUB whose container is malformed falls back to the first .opf in the archive, because a converter that wrote a bad container often still wrote a usable package.

Identifiers come back under the scheme they are: urn:isbn:… and a bare ISBN both become isbn, a UUID becomes uuid, anything else other. PDF dates are reduced to the day where they parse and kept verbatim where they do not, because a date nobody can read is still evidence.

A Kindle book's title is EXTH 503 where it is there, then the MOBI header's full name, then the Palm database name (cut to 31 bytes, with a note). Authors (100) and subjects (105) are every record of their kind, in order; publisher (101), description (103), date (106) and language (524) the first. EXTH 104 is an isbn when it is shaped like one and other when not. 113 is filed by its shape: a uuid when it is one (calibre writes its book id there), an asin when it is ten letters and digits, and other when it is neither. A contributor (108) is not an author, the same as an EPUB's dc:contributor, and is not reported. With no EXTH language, the language comes from the header's locale. Strings are decoded as the header says, UTF-8 or Windows-1252. A book whose text is encrypted still has its header read, and carries the note drm.

Every offset in a Kindle book is checked before it is followed: headers that are cut short or point outside the file give an empty record with a note, and an EXTH block that breaks off keeps the records before the break.

Covers

Metadata::cover is the cover image the file carries, with its media type read off its first bytes: JPEG, PNG, GIF, WebP or BMP. It is for library callers and is not part of the JSON record, which is a line of text.

A Kindle book's cover is EXTH 201, or the thumbnail, 202, when no cover is named. With neither, it is the record at the header's first image index, as calibre does, and only that record: when it is not a JPEG, PNG or GIF there is no cover, because the records after it can be illustrations, and kindlegen sometimes puts a font there first.

An EPUB's is the first of these that reaches an image:

1. The manifest item whose properties include cover-image (EPUB 3).
2. The item <meta name="cover" content="…"> names (EPUB 2). Some converters write the image's address there rather than its id, and that is followed too.
3. The guide's <reference type="cover">, a page, and the first <img src> or SVG <image href> on it. A guide that points straight at an image is taken at its word.
4. Any image in the manifest whose id or address says cover.

A route that names something unusable — not in the archive, not an image — leaves a note, and the next route is asked. Addresses are resolved against the document they are written in, with their fragment off and their percent-escapes undone, and one that climbs out of the archive or names a scheme (http:, data:) is not followed.

What the bytes say beats what the manifest says: a JPEG listed as image/png comes back as image/jpeg, with a note. A cover larger than bookmeta::MAX_COVER (16 MiB) is left out with a note, in either format, and an EPUB's is never decompressed further than that, whatever size the archive claims for it.

Cover::guessed says the file left the choice open: a Kindle book's first image record, or an EPUB image picked because its name says cover. Either can be an illustration from inside the book, so a caller that can get a cover another way, from a register by ISBN, should prefer that one. A cover the file names is never a guess. Cover::extension gives the usual file extension for its media type.

Asking a model

Some files say nothing useful. --prompt writes the question and sends nothing — where it goes is your business, your model, your machine:

$ bookmeta --prompt --excerpt opening.txt scan.pdf
You are cataloguing a book. Below is what its file says about itself, and the
opening of the text where one was available.

Known:
  path: scan.pdf
  title: Microsoft Word - ch3.doc

Missing: authors, publisher, date, language
…
Reply with one JSON object and nothing else. … Use null for anything the text
does not settle — a guess is worse than an absence, because nobody downstream
can tell them apart.

Then put it wherever you like and hand the answer back:

bookmeta --prompt scan.pdf | ollama run qwen3.5:9b | bookmeta --merge scan.pdf

Four things make this safe to depend on:

• Only what was missing is filled. A model that volunteers a better title for a book whose title was already known is overruled: what the file says is evidence, what a model says is a guess.
• A null is an absence, not a value. So is an empty string.
• The prompt is versioned and printed where you can read it. A record filled this way carries a note saying which prompt version did it.
• A reply that is not one JSON object is refused, not half-applied. Prose and code fences around the object are fine.

Nothing here reaches the network, so this tool keeps the contract whole: same input, same output, every run and every machine.

As a library

let meta = bookmeta::read(std::path::Path::new("philokalia.epub"));
assert_eq!(meta.title.as_deref(), Some("The Philokalia, Volume 3"));

if let Some(prompt) = bookmeta::prompt::compose(&meta, Some(opening)) {
    // send it wherever you like, then:
    // bookmeta::prompt::merge(&mut meta, &reply)?;
}

bookmeta::epub, bookmeta::pdf and bookmeta::mobi read one format each, and Metadata::missing says which fields a record still wants.

See also

• booksniff — what the file actually is.
• bibclean — whether what it says is any good.
• bibfetch — what a register says instead.