Architecture

Libris is a self-hosted book management system. It ingests ebook files, enriches them with metadata from multiple sources, organizes them into a structured library, and serves them via OPDS for e-readers. It syncs reading progress with KoReader via KoSync.

Monorepo Structure

libris/
├── apps/web/              # Vue 3 + Vite 8 SPA frontend (port 3100)
├── apps/obsidian-plugin/  # Read-only Obsidian companion (mirrors catalog into a vault)
├── services/api-hono/     # Hono backend (port 3000)
│   ├── src/types/         # Shared TypeScript types + Zod schemas
│   ├── src/lib/metadata/  # Book metadata extraction & API clients
│   └── src/lib/queue/     # BullMQ queue constants
├── docs/                  # VitePress documentation site (@libris/docs)
├── tests/e2e/             # Playwright end-to-end tests
├── scripts/               # Build & test scripts
├── .forgejo/workflows/    # CI/CD pipelines
└── docker-compose.test.yml

The Obsidian plugin is a third client of the API (alongside the SPA and KOReader/kosync). It only consumes GET /api/library* endpoints with a bearer API key — no write-path. See docs/obsidian-plugin.md for the layout it produces in the vault.

Workspace management: pnpm workspaces, orchestrated by Turborepo.

Tech Stack

Layer	Technology
Frontend	Vue 3 + Vite 8 SPA, vue-router 5, Nuxt UI v4, Tailwind CSS v4
Backend	Hono, @hono/zod-openapi
Database	PostgreSQL 17, Drizzle ORM
Job Queue	BullMQ, Redis 7
File Watching	Chokidar
Metadata Sources	Hardcover GraphQL API (requires API token via Settings)
Auth	bcrypt-hashed API keys, httpOnly session cookies
Testing	Playwright (E2E), Vitest (unit)
Toolchain	Turborepo, Vitest, Oxlint, Oxfmt
CI/CD	Forgejo Actions

Design Decisions

Why Hono

Hono with @hono/zod-openapi gives us three things in one:

1. Typed RPC client — Routes are defined with Zod schemas. The hc client (hono/client) infers request/response types from those schemas automatically. The frontend gets fully typed API calls with zero codegen, no hand-written interfaces, and no runtime overhead (~2KB).

2. OpenAPI docs for free — The same Zod schemas that power the typed client also generate the OpenAPI spec. Scalar UI at /_docs/scalar stays in sync with the actual code by definition.

3. Self-contained bundle — tsdown bundles the entire server into a single ~2MB file (dist/index.mjs). The Docker image has no node_modules — just the bundle and migrations.

The frontend uses hc directly in pages:

const client = useApiClient();
const { data } = await useAsyncData("library", async () => {
  const res = await client.api.library.$get({ query: { page: 1 } });
  if (!res.ok) throw new Error("Failed");
  return res.json(); // fully typed from the Zod schema
});

Things that hc can't handle use standalone composables: useUpload() for XHR file uploads with progress tracking, useServerEvents() for realtime streaming over a WebSocket opened at /api/events.

Why SPA

This is a private, self-hosted app behind authentication. Every page requires login — there are no public pages, no SEO concerns, no crawlers.

vite build produces pure static files in apps/web/dist/. In production, Hono serves these directly — no separate nginx container or Node.js runtime for the frontend. The build output is also ready for a native app shell (Capacitor/Tauri) in the future.

Auth uses httpOnly cookies set by the Hono API directly. Since the SPA and API are served from the same origin, cookies work without any cross-domain configuration.

How the Pieces Connect

graph TB
    subgraph Clients
        Browser["🌐 Browser"]
        EReader["📖 E-Reader\n(KOReader, Calibre)"]
    end

    subgraph Frontend
        SPA["Vue SPA\n:3100"]
    end

    subgraph Backend
        API["Hono API\n:3000"]
        Chokidar["Chokidar\nFile Watcher"]
    end

    subgraph Infrastructure
        PG["PostgreSQL\n(Drizzle ORM)"]
        Redis["Redis\n(BullMQ + KV + Pub/Sub)"]
        FS["File System\n(inbox / library)"]
    end

    Browser -->|Cookie auth| SPA
    SPA -->|API calls| API
    API -->|WebSocket| Browser
    EReader -->|OPDS| API
    EReader -->|KoSync| API
    Chokidar -->|New file detected| API
    API --> PG
    API --> Redis
    API --> FS

1. Browser loads the Vue SPA and makes API calls directly to the Hono backend
2. Hono API handles auth via httpOnly cookies or API keys and manages all business logic: REST endpoints, job processing, file management
3. Chokidar watches the inbox directory and enqueues jobs when files appear
4. BullMQ workers process jobs: detect → parse → fetch metadata → organize

If parsing cannot extract any usable metadata, the pipeline stops before external lookup and the book is moved to manual review instead of issuing an ambiguous fallback search. 5. Event bus bridges Redis pub/sub to WebSocket clients via a local EventEmitter. The publisher reuses the shared ioredis instance; the subscriber uses a dedicated connection (required by Redis SUBSCRIBE mode). Reconnection uses exponential backoff (200ms–10s) with automatic resubscribe 6. Redis connections are consolidated through a single shared ioredis instance (getSharedRedis()). BullMQ queues, the KV store, the cache, and the event-bus publisher all share this one connection. Only BullMQ workers (which need blocking reads) and the event-bus subscriber (which needs SUBSCRIBE mode) open their own connections — ~8 total in production instead of ~20 7. OPDS serves the organized library to e-readers (Calibre, KOReader, etc.) 8. KoSync syncs reading progress between KoReader devices

Backend: Single-Process Monolith

The API server runs everything in one Hono process. All bootstrap logic lives in a single file (src/bootstrap.ts) that runs each step in order:

1. Validate directory access — checks that LIBRIS_INBOX_PATH and LIBRIS_LIBRARY_PATH are readable/writable (skipped in test)
2. Run migrations — applies Drizzle migrations before accepting requests (skipped in test — tests use PGlite with manual migration)
3. Create DB singleton — initializes the Drizzle database connection
4. Setup KV stores — Redis-backed in production, in-memory in dev/test
5. Create BullMQ queues — reuses the shared ioredis instance (stubbed in test)
6. Start inbox file watcher — Chokidar watches LIBRIS_INBOX_PATH for new files (skipped in test)
7. Start BullMQ workers + schedulers — initializes workers for all pipeline and scheduled queues (skipped in test)
8. Register shutdown handler — graceful teardown of workers, queues, watcher, event bus, Redis, and DB

Workers

In addition to the ingestion pipeline queues, scheduled workers handle external sync and maintenance:

Worker	Schedule	Purpose
hardcover-sync	Daily 4 AM	Bidirectional reading-status sync with Hardcover for all linked books (push: Libris → Hardcover; pull: Hardcover → reading_aggregate.external_status). Gated on hardcover.syncEnabled setting; ISBN matching gated on hardcover.metadataEnabled.
progress-history-cleanup	Daily 3 AM	Cleans up reading progress history older than 1 year

Scheduled Tasks

Scheduled tasks run independently of BullMQ:

Task	Schedule	Purpose
db:cleanup-orphaned-files	Daily 3 AM	Cleans up orphaned files with no book records
jobs:cleanup-completed	Hourly	Removes completed jobs from queues

Job & Queue Management API

The /api/jobs routes expose BullMQ internals for the admin UI. Queue listings cover every queue in the registry — the ingestion pipeline queues (book-*), scheduler queues (hardcover-sync, progress-history-cleanup), and the db-maintenance queue. The same aggregation backs /api/settings/status for the admin diagnostics panel. The home dashboard's "pipeline" indicator is intentionally scoped to the ingestion queues only, since it reports on book ingestion flow rather than overall queue health.

Method	Path	Purpose
GET	/api/jobs	List jobs across all queues (paginated, filterable)
GET	/api/jobs/status	Job counts per queue (waiting/active/completed/etc.)
GET	/api/jobs/failed	List failed jobs across all queues
GET	/api/jobs/{id}	Full job detail (payload, timestamps, progress)
GET	/api/jobs/{id}/logs	Job log lines stored via job.log()
POST	/api/jobs/{id}/retry	Retry a failed job
POST	/api/jobs/queues/{name}/pause	Pause a queue
POST	/api/jobs/queues/{name}/resume	Resume a paused queue
POST	/api/jobs/queues/{name}/clean	Remove all failed jobs from a queue
POST	/api/jobs/queues/{name}/drain	Remove all waiting/delayed jobs from a queue

Reading Status

Reading status is derived from KoSync progress data rather than being set manually:

• unread — no progress recorded
• reading — progress between 0% and 100%
• finished — progress at 100%
• paused — no progress update for a configurable period

Frontend: SPA

The Vue app runs as a Single Page Application (SPA):

• The Hono API sets httpOnly auth cookies directly — no BFF proxy layer
• All API calls go directly from the browser to the Hono backend
• Authentication is handled via httpOnly cookies set by the API's /api/auth/* endpoints
• CORS is configured on the API to allow requests from the SPA origin

Multi-User Auth

Auth Model

Authentication is API key-based. During initial setup (POST /api/auth/setup), the first key is created and marked as admin. Admin users can create additional keys via POST /api/auth/keys. Keys are 32-byte random hex strings, bcrypt-hashed at rest.

Clients authenticate via Authorization: Bearer <key> header or httpOnly session cookie (set at login). Route-level auth policies are declared in a lookup table (route-policy.ts) — first match wins:

Policy	Applied to	Behavior
public	/api/auth/setup, /api/auth/login, etc.	No authentication
optional	/api/health	Enriched response when authenticated
api-key	/api/* (default)	Requires valid API key
admin	/api/jobs/*	Requires valid API key with isAdmin flag
opds	/opds/*	Basic auth against per-user service credentials
kosync	/kosync/*	Header-based auth (x-auth-user / x-auth-key)

Book Ownership

Each book tracks its uploader via books.created_by (foreign key to api_keys.id). Upload attribution flows through the upload_registry table, which deduplicates by file checksum so re-uploads of the same file are ignored.

requireBookOwnership(c, db, bookId) enforces access control on mutations: only the uploading user or an admin can modify or delete a book. Unowned books (legacy data with created_by = null) are admin-only.

Data Isolation

Per-user data is scoped by API key ID:

• Reading progress — KoSync progress records are tied to the authenticated key
• Service credentials — OPDS and KoSync credentials in service_credentials are linked to an api_key_id
• Stats and streaks — Dashboard data (reading stats, streak counts) are computed per user
• Hardcover sync — External service credentials and sync state are per user

Shared data (the book catalog, library organization, metadata) is visible to all authenticated users.

Auth Caching

The auth middleware uses in-memory TtlCache instances (5-minute TTL, max 500 entries) to avoid running bcrypt on every request:

• apiKeyCache — caches Bearer/session key verification results (keyed by SHA-256 of the raw key)
• opdsCache — caches OPDS Basic auth verification results

Any operation that changes key validity or privileges (deletion, role changes, credential rotation) must call clearAuthCaches() to prevent stale authorization data.

Rate Limiting

Three tiers, enforced per client IP via Redis (with in-memory fallback for auth-critical tiers):

Tier	Limit	Applied to
auth	10 req/min	/api/auth/*, /opds/*, /kosync/users/auth
keyCreation	5 req/hour	POST /api/auth/setup, POST /api/auth/keys
general	100 req/min	All other /api/* and /kosync/* routes

IP extraction (getRequestIp) reads the direct connection address by default. When TRUST_PROXY_HEADERS=1 is set, it prefers X-Real-IP or the first entry in X-Forwarded-For — use this when running behind a reverse proxy. Rate limiting is disabled in development and test environments.

Book Ingestion Pipeline

See the API Reference for endpoint details.

flowchart TD
    A["📁 File appears in inbox"] --> B["BOOK_DETECTED"]
    B --> |"Compute checksum, detect format,\ncreate DB records, dedup"| C["BOOK_PARSE_FILE"]
    C --> |"Extract metadata from EPUB/PDF\n(Dublin Core, XMP, PDF Info)"| D["BOOK_FETCH_METADATA"]
    D --> |"Query Hardcover\nInsert candidates, detect duplicates\nSet status → review"| E["👤 User reviews candidates"]
    E --> |"Pick fields from sources, approve"| F["BOOK_ORGANIZE"]
    F --> |"Move to /library/Author/Title/\nDownload cover, embed metadata in EPUB\nCompute MD5, set status → organized"| G["✅ Organized"]
    G -.-> |"Refetch metadata\n(POST /api/library/id/refetch)"| D
    G -.-> |"Re-organize\n(POST /api/library/id/reorganize)"| F

All jobs retry 3 times with exponential backoff (1s base). Completed jobs are automatically removed after 1,000 entries or 24 hours (removeOnComplete), and failed jobs after 1,000 entries or 7 days (removeOnFail), preventing unbounded Redis memory growth. Each worker has a tuned lockDuration (30s–10min depending on expected job time) and maxStalledCount: 2 so that hung jobs are detected and recovered by BullMQ's stalled job checker. The hardcover sync worker checkpoints progress, allowing it to resume from where it left off after a crash.

Hardcover Reading-Status Sync

The hardcover-sync worker is bidirectional for status:

• Push (Libris → Hardcover) — for each user, find books whose computed status or progress drifted from hardcover_sync_log.last_status/last_progress and upsert via insert_user_book + insert_user_book_read. Books with no local reading data are not pushed (so we don't clobber Hardcover with unread).
• Pull (Hardcover → Libris) — for each user, fetch the full me.user_books list, map status_id → ReadingStatus (1=unread, 2=reading, 3=finished, 4=paused, 5(DNF)=paused), and upsert reading_aggregate.external_status. This lets a fresh Libris install instantly reflect the user's existing Hardcover library on connect.

The effective reading status used by the API and UI follows this precedence:

manual_status                                        -- user override (also pushed)
?? local-computed (only if any local reading_progress for this book)
?? external_status                                   -- Hardcover-pulled fallback
?? "unread"

external_status is read-only from the push side — it never feeds back outward, which prevents a sync loop. Only manual_status represents authoritative user intent that flows in both directions (the outward push of manual_status is tracked separately, see issue libris-wfmj).