DocuVault

Version 0.1.0 — Alpha

Not production-ready. DocuVault is functional for local and self-hosted use but has not been audited or hardened for public internet exposure. APIs, environment variables, and the database schema may change without notice until a stable 1.0 release is declared.

A self-hosted, multi-user document management platform with AI-powered topic classification, pluggable cloud storage backends, and a security-first architecture.

---

Features

• Document management — upload PDF, DOCX, images, and plain text; full-text extraction stored in PostgreSQL
• AI classification — automatic topic tagging via LM Studio (local), Ollama, OpenAI, Anthropic, or any OpenAI-compatible provider (Groq, xAI, DeepSeek, OpenRouter, Gemini, Mistral)
• Folder organisation — hierarchical folders with move, rename, and delete; breadcrumb navigation
• Document sharing — share by user handle with view or edit permission; "Shared with me" virtual folder; per-recipient revocation
• Storage quota — per-user limit enforced atomically; amber/red quota bar at 80 % / 95 %; quota decremented on delete
• Cloud storage backends — connect OneDrive, Google Drive, Nextcloud, or any WebDAV server as a personal storage backend; credentials encrypted with HKDF per-user keys
• Auth — email/password registration with HIBP breach check, optional TOTP 2FA, 8–10 backup codes, password reset by email, sign-out-all-devices
• Admin panel — user CRUD, quota assignment, per-user AI provider override, AI provider system configuration, audit log viewer with CSV export
• Observability — structured JSON logging via structlog, per-request correlation IDs, Loki + Promtail + Grafana stack included in Docker Compose
• Production hardening — non-root containers, read-only filesystems, CSP/CSRF/Origin-validation middleware, per-IP and per-account rate limiting, CVE scanning gate

---

Architecture

┌─────────────────────────────────────────┐
│  Vue 3 frontend (Vite + Pinia + Tailwind) │
│  Port 5173                               │
└────────────────┬────────────────────────┘
                 │ REST / httpOnly cookie
┌────────────────▼────────────────────────┐
│  FastAPI backend (Python 3.12)          │
│  Port 8000                              │
│  ├── api/auth.py      (JWT + TOTP)      │
│  ├── api/documents.py                   │
│  ├── api/folders.py                     │
│  ├── api/shares.py                      │
│  ├── api/cloud.py     (cloud backends)  │
│  ├── api/admin.py                       │
│  └── api/audit.py                       │
└──┬──────────┬──────────┬───────────────┘
   │          │          │
   ▼          ▼          ▼
PostgreSQL  MinIO     Redis
(data)    (objects) (rate limit / TOTP replay)
              │
              └── Celery worker
                  (extract + classify tasks)
                  Celery beat
                  (daily audit export)

Storage object key schema: {user_id}/{document_id}/{uuid4()}{ext} — human filenames live in PostgreSQL only.

Token flow: Access token (15 min, ES256) in Pinia memory only. Refresh token (30 day) in an httpOnly SameSite=Strict cookie. Refresh rotation on every use; reuse revokes the entire token family.

---

Stack

Layer	Technology
Backend	Python 3.12, FastAPI 0.136+, SQLAlchemy 2.0 async, psycopg v3, Alembic
Task queue	Celery 5 + Redis, celery-beat for scheduled tasks
Object store	MinIO (S3-compatible)
Database	PostgreSQL 17
Frontend	Vue 3 (Options API), Pinia, Vue Router 4, Vite, Tailwind CSS
Auth	PyJWT (ES256), pwdlib[argon2], pyotp (TOTP), cryptography (HKDF/AES-GCM)
AI providers	Anthropic, OpenAI, Ollama, LM Studio, GenericOpenAI (Groq/xAI/DeepSeek/Gemini/…)
Cloud backends	Google Drive, OneDrive (MSAL), Nextcloud, WebDAV
Observability	structlog, Loki, Promtail, Grafana
Rate limiting	slowapi (per-IP + per-account)
Load testing	Locust (backend/load_tests/)

---

Prerequisites

• Docker and Docker Compose v2
• openssl (for generating secrets)
• (Optional) LM Studio or Ollama running on the host for local AI classification

---

Quick Start

# 1. Clone
git clone <repo-url>
cd document_scanner

# 2. Create .env from the example
cp .env.example .env

# 3. Generate required secrets
python3 -c "import secrets; print(secrets.token_hex(64))"   # → SECRET_KEY
python3 -c "import secrets; print(secrets.token_urlsafe(32))" # → CLOUD_CREDS_KEY

# 4. Edit .env — at minimum set:
#   SECRET_KEY, CLOUD_CREDS_KEY, ADMIN_EMAIL, ADMIN_PASSWORD
#   (everything else has usable defaults for local dev)
nano .env

# 5. Start all services
docker compose up -d --build

# 6. Run database migrations
docker compose exec backend alembic upgrade head

# 7. Open the app
open http://localhost:5173

The bootstrap admin account (email + password from .env) is created automatically on first startup if no users exist.

---

Service URLs

Service	URL	Notes
Vue frontend	http://localhost:5173	Main app
FastAPI backend	http://localhost:8000	REST API
API health	http://localhost:8000/health	Postgres + MinIO probe
OpenAPI docs	http://localhost:8000/docs	Swagger UI (dev)
MinIO console	http://localhost:9001	Object browser
Grafana	http://localhost:3000	Log dashboard (admin/changeme)
Loki	http://localhost:3100/ready	Log aggregator

---

Environment Variables

Copy .env.example to .env. Only the fields marked Required must be set before first boot. See RUNBOOK.md for the full reference.

Required

Variable	Description
SECRET_KEY	JWT signing secret — generate with openssl rand -hex 32
CLOUD_CREDS_KEY	Master key for cloud credential encryption — generate with openssl rand -hex 16 (pad to 32 chars)
ADMIN_EMAIL	Bootstrap admin email
ADMIN_PASSWORD	Bootstrap admin password (must pass strength check)
POSTGRES_PASSWORD	PostgreSQL superuser password
MINIO_ROOT_PASSWORD	MinIO root password
REDIS_PASSWORD	Redis requirepass password

Optional (sensible defaults for local dev)

Variable	Default	Description
CORS_ORIGINS	http://localhost:5173	Comma-separated allowed origins
SMTP_HOST	(unset)	Leave empty to log password-reset links to stdout
LOG_JSON	false	Set true in production for structured JSON logs
DEFAULT_AI_PROVIDER	ollama	Used on first boot seed; overridable per-user in admin panel
GOOGLE_CLIENT_ID/SECRET	(unset)	Required only if using Google Drive backend
ONEDRIVE_CLIENT_ID/SECRET	(unset)	Required only if using OneDrive backend

---

Development

Backend (local, no Docker)

cd backend
pip install -r requirements.txt -r requirements-dev.txt
# Requires a running PostgreSQL and Redis — use docker compose up postgres redis minio
uvicorn main:app --reload --port 8000

Frontend (local, no Docker)

cd frontend
npm install
npm run dev          # http://localhost:5173

Database migrations

# Apply all pending migrations
docker compose exec backend alembic upgrade head

# Create a new migration after changing db/models.py
docker compose exec backend alembic revision --autogenerate -m "describe change"

Live reload in Docker

docker-compose.yml mounts ./backend:/app so Python changes reload automatically. After changing requirements.txt or Dockerfile:

docker compose build backend celery-worker celery-beat
docker compose up -d backend celery-worker celery-beat

---

Testing

# Backend — all tests
cd backend && pytest -v

# Backend — integration tests (requires live PostgreSQL + MinIO)
INTEGRATION=1 pytest -v

# Frontend
cd frontend && npm run test

# Load testing (requires a running stack + load-test user)
cd backend
pip install locust
locust -f load_tests/locustfile.py --host http://localhost:8000

Test coverage by layer

Layer	Approach
Service / business logic	Unit tests with mocked dependencies
DB queries	Integration tests against real PostgreSQL
API endpoints	httpx.AsyncClient against a real DB
Auth flows	Full round-trip (register → login → TOTP → refresh → revoke)
Security invariants	Negative tests (wrong owner → 403/404, admin block, token replay)
Frontend	Vitest unit tests in frontend/tests/

---

AI Providers

AI provider settings live in the system_settings database table and are managed through the Admin → AI Providers panel. API keys are encrypted at rest with HKDF/AES-GCM.

Provider	Type	provider_id
LM Studio	Local (OpenAI-compat)	lmstudio
Ollama	Local (OpenAI-compat)	ollama
OpenAI	Cloud	openai
Anthropic	Cloud (native)	anthropic
Groq	Cloud (OpenAI-compat)	groq
xAI Grok	Cloud (OpenAI-compat)	xai
DeepSeek	Cloud (OpenAI-compat)	deepseek
OpenRouter	Cloud (OpenAI-compat)	openrouter
Gemini (compat)	Cloud	gemini
Mistral (compat)	Cloud	mistral

Documents whose classification fails are automatically retried by Celery at 30 s, 90 s, and 270 s. After three failures the document shows a red "Classification failed" badge with a "Re-analyze" button.

---

Cloud Storage Backends

Users connect cloud storage through Settings → Cloud Storage. Credentials are encrypted per-user with an HKDF-derived key and never returned in API responses.

Provider	Auth method	Notes
Google Drive	OAuth 2.0	Requires GOOGLE_CLIENT_ID/SECRET
Microsoft OneDrive	OAuth 2.0 (MSAL)	Requires ONEDRIVE_CLIENT_ID/SECRET
Nextcloud	Username + password	Custom server URL; SSRF allowlist enforced
WebDAV	Username + password	Any RFC 4918 server; SSRF allowlist enforced

Connection statuses: ACTIVE, REQUIRES_REAUTH, ERROR. An externally revoked OAuth token transitions to REQUIRES_REAUTH without a 500 error.

---

Security Highlights

• Passwords hashed with Argon2id; HIBP breach check on registration and password change
• JWT signed with ES256 (ECDSA P-256); access token 15 min; refresh token 30 days in httpOnly SameSite=Strict cookie
• Token fingerprinting — access token embeds HMAC of User-Agent + Accept-Language; validated on every request
• TOTP replay prevention — used codes marked in Redis within the 90 s validity window
• Quota enforced by a single atomic UPDATE … RETURNING — no read-then-write race
• IDOR protection — every resource endpoint asserts resource.user_id == current_user.id
• SSRF prevention — user-supplied WebDAV/Nextcloud URLs pass a hostname allowlist
• CSP / security headers on every response
• Container hardening — non-root user (appuser uid 1000), read-only root filesystem, all Linux capabilities dropped
• Audit log — all auth events, quota violations, and admin actions logged without document content

Security gate (run before every production deploy):

bandit -r backend/                                   # zero HIGH
pip audit                                            # zero critical/high CVEs
npm audit --audit-level=high                         # zero high/critical
docker scout cves local://docuvault-backend:latest --only-severity critical --exit-code

See SECURITY.md for the full threat model and RUNBOOK.md for operational procedures.

---

Observability

Structured JSON logs are emitted to stdout and collected by Promtail → Loki → Grafana.

Every request gets an X-Correlation-ID header bound to all log lines for that request. Log fields include correlation_id, path, method, status_code, duration_ms, and user_id (when authenticated).

Access Grafana at http://localhost:3000 (default credentials: admin / changeme — change via GRAFANA_ADMIN_PASSWORD in .env). Add a Loki datasource pointing to http://loki:3100 to query logs.

---

Project Structure

.
├── backend/
│   ├── ai/              # Provider implementations + classifier
│   ├── api/             # FastAPI routers (auth, documents, folders, shares, cloud, admin, audit)
│   ├── db/              # SQLAlchemy models + async session
│   ├── deps/            # FastAPI dependency injection (auth, db, utils)
│   ├── migrations/      # Alembic migration versions
│   ├── services/        # Business logic (auth, classifier, storage, email, audit, rate limiting)
│   ├── storage/         # StorageBackend ABC + MinIO/Google/OneDrive/Nextcloud/WebDAV backends
│   ├── tasks/           # Celery tasks (document processing, email, audit export)
│   ├── load_tests/      # Locust load test scripts
│   ├── tests/           # pytest test suite
│   ├── main.py          # FastAPI application factory + middleware
│   └── config.py        # Pydantic settings
├── frontend/
│   └── src/
│       ├── api/         # Axios API client modules
│       ├── components/  # UI components (documents, folders, cloud, admin, sharing, layout)
│       ├── stores/      # Pinia stores (auth, documents, folders, cloud connections)
│       ├── views/       # Page-level view components
│       └── utils/       # formatters.js (formatDate, formatSize, providerColor, …)
├── docker/
│   ├── postgres/        # initdb.d — user + role bootstrap SQL
│   └── loki/            # Loki + Promtail config
├── docker-compose.yml
├── .env.example
├── RUNBOOK.md           # Operational runbook (env vars, backup, on-call, failure modes)
└── SECURITY.md          # Threat model and security gate results

---

Contributing

1. All new endpoints require at least one integration test
2. Run pytest -v and npm run test — zero failures required
3. Run bandit -r backend/ — zero HIGH findings
4. No raw SQL string interpolation; no pickle; no eval; no innerHTML with user data
5. Service layer raises ValueError (never HTTPException); only the router layer raises HTTPException
6. Before writing a new helper, check the shared module map in CLAUDE.md