bookmeta: read MOBI, AZW and AZW3 from the MOBI header and its EXTH records

A Kindle book's title, authors, publisher, description, ISBN, ASIN,
subjects, date and language come from the EXTH block, with the full name,
the Palm database name and the header's locale as fallbacks. The cover
image (EXTH 201, or the thumbnail) is returned as Metadata::cover with its
media type; it stays out of the JSON record. Every offset is bounds-checked:
a damaged header gives an empty record with a note, a broken EXTH block
keeps what came before the break, and an encrypted book is still read and
noted as drm.

Metadata gains description, which the EPUB reader now fills from
dc:description as well.

Fixtures are a story and cover written for the tests (CC0), converted with
calibre to MOBI 6 and AZW3, plus no-EXTH, DRM-flagged and truncated copies.
This commit is contained in:
funkpower committed 2026-11-13 14:17:44 -05:00
1 parent 6f32f52293
commit b7d98e9d1a
13 files changed
+955 -3

No files matched your search

+28
View File
@@ -0,0 +1,28 @@
# Fixtures
## MOBI and AZW3
`lantern.html` is a two-paragraph story written for these tests, and
`lantern-cover.jpg` a 120×180 cover drawn for them with Pillow. Both are
dedicated to the public domain (CC0 1.0), and so is everything made from them.
`lantern.mobi` (MOBI 6) and `lantern.azw3` (KF8) were made from them with
calibre 9.14:
```sh
common=(--title "A Lantern in the Orchard" --authors "Ada Quill & Tomás Reed"
--publisher "Orchard Press" --isbn 9780306406157 --tags "Fiction,Villages"
--language en --pubdate 2001-04-05 --comments "A short story written for testing."
--cover lantern-cover.jpg)
ebook-convert lantern.html lantern.mobi --mobi-file-type old "${common[@]}"
ebook-convert lantern.html lantern.azw3 "${common[@]}"
```
The other three are `lantern.mobi` with one thing broken. `r0` is the first
record's offset, the big-endian 32-bit number at byte 78 (136 here).
| File | Change |
|---|---|
| `lantern-no-exth.mobi` | The EXTH flag (`0x40`) cleared in the MOBI header's flags, at `r0 + 16 + 112` |
| `lantern-drm.mobi` | The PalmDOC encryption type, at `r0 + 12`, set to 2 |
| `lantern-truncated.mobi` | The first 1024 bytes only |
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+5
View File
@@ -0,0 +1,5 @@
<html><head><meta charset="utf-8"><title>x</title></head><body>
<h1>A Lantern in the Orchard</h1>
<p>The lantern hung from the lowest branch of the pear tree, and every evening someone lit it. Nobody in the village could say who.</p>
<p>Café owners swore it was the miller; the miller swore it was the wind.</p>
</body></html>
Binary file not shown.
+102
View File
@@ -0,0 +1,102 @@
//! MOBI and AZW3 files as a converter writes them, and as they break.
use std::path::{Path, PathBuf};
fn fixture(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.join("tests/fixtures")
.join(name)
}
fn read(name: &str) -> bookmeta::Metadata {
bookmeta::read(&fixture(name))
}
/// What the two converted fixtures were given, and so must say.
fn says_what_it_was_given(meta: &bookmeta::Metadata) {
assert_eq!(meta.source, "mobi");
assert_eq!(meta.title.as_deref(), Some("A Lantern in the Orchard"));
assert_eq!(meta.authors, ["Ada Quill", "Tomás Reed"]);
assert_eq!(meta.publisher.as_deref(), Some("Orchard Press"));
assert_eq!(
meta.description.as_deref(),
Some("A short story written for testing.")
);
assert_eq!(meta.date.as_deref(), Some("2001-04-05T04:00:00+00:00"));
assert_eq!(meta.language.as_deref(), Some("en"));
assert_eq!(meta.subjects, ["Fiction", "Villages"]);
assert_eq!(meta.identifiers["isbn"], "9780306406157");
assert!(meta.identifiers.contains_key("asin"));
let cover = meta.cover.as_ref().expect("a cover");
assert_eq!(cover.media_type, "image/jpeg");
let original = std::fs::read(fixture("lantern-cover.jpg")).expect("the cover fixture");
assert_eq!(cover.bytes, original);
}
#[test]
fn a_mobi_6_file_says_what_it_was_given() {
let meta = read("lantern.mobi");
says_what_it_was_given(&meta);
assert!(meta.notes.is_empty(), "{:?}", meta.notes);
}
#[test]
fn an_azw3_file_says_what_it_was_given() {
let meta = read("lantern.azw3");
says_what_it_was_given(&meta);
assert!(meta.notes.is_empty(), "{:?}", meta.notes);
}
#[test]
fn a_file_with_no_exth_still_has_its_full_name_and_locale() {
let meta = read("lantern-no-exth.mobi");
assert_eq!(meta.source, "mobi");
assert_eq!(meta.title.as_deref(), Some("A Lantern in the Orchard"));
assert_eq!(meta.language.as_deref(), Some("en"));
assert!(meta.authors.is_empty());
assert!(meta.identifiers.is_empty());
assert_eq!(meta.cover, None);
}
#[test]
fn a_truncated_file_is_an_empty_record_with_a_reason() {
let meta = read("lantern-truncated.mobi");
assert_eq!(meta.source, "none");
assert!(meta.is_empty());
assert_eq!(meta.notes, ["the first record lies outside the file"]);
}
#[test]
fn a_drm_flagged_file_has_its_metadata_and_says_drm() {
let meta = read("lantern-drm.mobi");
says_what_it_was_given(&meta);
assert_eq!(meta.notes, ["drm"]);
}
#[test]
fn every_truncation_of_the_mobi_fixture_returns() {
let bytes = std::fs::read(fixture("lantern.mobi")).expect("the fixture");
let first_record_end = u32::from_be_bytes(bytes[86..90].try_into().unwrap()) as usize;
for len in 0..=bytes.len() {
let meta = bookmeta::from_bytes("cut.mobi", &bytes[..len]);
if len < first_record_end {
assert!(meta.is_empty(), "cut at {len}: {meta:?}");
}
}
}
#[test]
fn every_single_byte_corruption_of_the_header_returns() {
let bytes = std::fs::read(fixture("lantern.mobi")).expect("the fixture");
// The Palm header, the record list, and the whole of the first record:
// every offset the reader follows is in there.
let first_record_end = u32::from_be_bytes(bytes[86..90].try_into().unwrap()) as usize;
for at in 0..first_record_end {
for value in [0x00, 0x7f, 0xff] {
let mut broken = bytes.clone();
broken[at] = value;
let _ = bookmeta::from_bytes("broken.mobi", &broken);
}
}
}