# log — Formation Recipe (FvW v8 §38)

**Module ID:** `log`
**Module name:** Log Module
**Lifecycle phase:** formation
**Status:** INTENT ONLY. No code yet.
**Date:** 2026-07-21

---

## 1. Intent (the "why")

The log module provides centralized, structured logging infrastructure for FreshCloud Mail. It solves three critical problems:
- **Audit trails** for compliance, security investigations, and per-user action history
- **Diagnostic visibility** for developers during operation (errors, warnings, info)
- **Performance monitoring** through structured metrics (write rates, buffer sizes, sync lag)

It acts as the system's "nervous system," capturing events for debugging, auditing, and observability. **It IS the audit system** — every audit record in the application flows through this module.

This is **intentionally different from MailSo's log module**, which was a generic logging utility. FreshCloud's `log` module is the **dedicated audit + observability subsystem**.

---

## 2. User-facing behaviour

A user (or other module) can:
- **Write a log event** with severity (DEBUG, INFO, WARNING, ERROR, CRITICAL)
- **Record a user action** (audit event) with structured context
- **Configure level filtering** via config (e.g. suppress DEBUG in production)
- **Force flush** the buffer to PB on shutdown or critical events
- **Read recent activity** for a user (UI activity feed)
- **Read recent errors** for a user (support page)

A developer integrating with the module:
- Calls `\FreshCloud\Mail\Log\Logger::write($level, $message, $context)` for general logs
- Calls `\FreshCloud\Mail\Log\Logger::audit($action, $context)` for user actions
- The Logger interface is the same one the `compose` module already uses

---

## 3. Public contract (data shapes + signatures)

Target namespace: `FreshCloud\Mail\Log\*` (NOT `MailSo\Log\*` — clean-room, MIT).

```typescript
// Interface
interface Logger {
    write(
        level: LogType,
        message: string,
        context?: Record<string, mixed>
    ): void;

    audit(
        action: string,
        context?: Record<string, mixed>
    ): void;

    flush(): void;

    getRecentActivity(
        userId: string,
        limit: int
    ): LogEntry[];

    getRecentErrors(
        userId: string,
        since: DateTime
    ): LogEntry[];
}

interface Driver {
    write(entry: LogEntry): void;
    flush(): void;
}

enum LogType {
    Debug = 0;
    Info = 1;
    Warning = 2;
    Error = 3;
    Critical = 4;
}

interface LogEntry {
    id: string;                  // UUID
    timestamp: DateTime;         // ISO 8601 UTC
    level: LogType;
    message: string;             // max 8192 chars
    context: Record<string, mixed>;
    user_id?: string;            // null for system-level events
    account_id?: string;         // null for system-level events
    action?: string;             // set if this is an audit event
    source: string;              // which module emitted this
}
```

---

## 4. Persistence (what gets stored where)

- **PocketBase `email_audit` table** (the canonical audit log, already deployed at collection `pbc_*`):
  ```sql
  -- 17 fields per formation §4 of compose
  CREATE TABLE email_audit (
    id TEXT PRIMARY KEY,           -- UUID
    user_id TEXT NOT NULL,         -- PB user
    account_id TEXT NOT NULL,      -- IMAP account
    action TEXT NOT NULL,          -- event name
    level TEXT NOT NULL,           -- LogType as string
    message TEXT NOT NULL,
    context_json TEXT NOT NULL,    -- JSON serialized context
    source TEXT NOT NULL,          -- module name
    created_at TEXT NOT NULL       -- ISO 8601 UTC
  );
  -- Indexes: (user_id, created_at DESC), (level, created_at DESC)
  ```

  **Note**: We use `email_audit` (not `audit_logs` or `logs`). The compose module's formation §4 already specified this. The `log` module writes here, the `compose` module's audit calls land here too. **One audit table, one source of truth.**

- **In-memory buffer** (transient, not persisted):
  - 10,000-entry circular buffer for `getRecentActivity()` / `getRecentErrors()` queries
  - Lost on process restart (acceptable; PB is the source of truth)
  - Flush threshold: 100 entries OR 5 seconds OR explicit `flush()` call

- **No local file logs in v0.1.0.** MailSo wrote to `/var/log/freshmail/*.log`. We deliberately skip this. The reasoning: PB is the source of truth, file logs add complexity (rotation, retention, disk usage), and operators can query PB instead. **If we add file logs later, the design intent for them goes in a separate formation-recipe.**

---

## 5. Dependencies (what this module uses, what uses this)

**Used by** (modules that call Logger):
- `compose` (email send/receive events)
- (future) `auth` (login/logout)
- (future) `accounts-ui` (account CRUD events)
- (future) `smtp` (send failures)
- (future) `imap-client` (fetch failures)
- (future) every module that emits audit events

**Dependencies** (what this module uses):
- `PocketBaseAdapter` interface (the abstraction we use everywhere; real impl wraps the joseescobar SDK)
- PHP 8.1+ (enums, readonly classes)
- `Ramsey\Uuid\Uuid` (already in composer.json for the compose module)

**No engine module dependencies.** This is the most foundational module.

---

## 6. Behaviour inventory (the 6 public methods)

| # | Method | Signature | Purpose | Side Effects |
|---|--------|-----------|---------|--------------|
| 1 | `write` | `void write(LogType $level, string $message, array $context = []): void` | General-purpose log | Pushes to all registered drivers |
| 2 | `audit` | `void audit(string $action, array $context = []): void` | User-action audit event | Inserts to PB `email_audit` table |
| 3 | `flush` | `void flush(): void` | Force buffer to PB | Inserts buffered entries to PB |
| 4 | `getRecentActivity` | `array getRecentActivity(string $userId, int $limit = 50): array` | UI activity feed | Reads from in-memory buffer (fast) |
| 5 | `getRecentErrors` | `array getRecentErrors(string $userId, \DateTimeInterface $since): array` | Support page | Reads from in-memory buffer (fast) |
| 6 | `addDriver` | `void addDriver(Driver $driver): void` | Register output target (future use) | None |

The compose module already uses methods 1 and 2 (via the `Logger` interface it imports).

---

## 7. State machine

A `LogEntry` is in one of 4 states:

```
pending (in memory buffer, not yet on PB)
   |
   | (flush threshold reached, or explicit flush(), or Critical level)
   v
persisted (in PB email_audit)
   |
   | (older than 90 days, retention policy)
   v
pruned (deleted from PB, only summary stats remain)
```

A `Logger` is in one of 3 modes:
- `online` (PB reachable, writes go directly to PB)
- `buffering` (PB unreachable, writes go to in-memory buffer, sync on reconnect)
- `failed` (PB unreachable AND buffer full, drops entries with a CRITICAL-level alert)

---

## 8. Side effects

For each public method:

| Method | Side Effects |
|---|---|
| `write()` | 1 in-memory append; if `level === Critical` AND PB is online, also 1 PB insert |
| `audit()` | 1 PB insert (always) |
| `flush()` | N PB inserts (one per buffered entry); clears buffer |
| `getRecentActivity()` | 0 (reads from buffer) |
| `getRecentErrors()` | 0 (reads from buffer) |
| `addDriver()` | 0 (registers driver in instance) |

Metrics emitted (per Sweep 3 observability):
- `freshcloud_mail_log_writes_total{level, status="ok|buffered|failed"}` — counter
- `freshcloud_mail_log_buffer_size` — gauge
- `freshcloud_mail_log_sync_lag_seconds` — gauge (oldest buffered entry age)

---

## 9. Input validation

- `LogType` must be a valid enum value (PHP enforces)
- `message` is truncated to 8192 chars; truncation is logged as WARNING
- `context` keys must be strings; values can be any serializable type
- `audit()` requires `action` to be a non-empty string; allowed values are an open enum (logged strings, not strict), but we recommend kebab-case verbs (`email-sent`, `email-received`, `login`, etc.)
- For `audit()`: if no `userId` is in the context, throw `AuthBoundaryViolation` (the call must include the user)
- For `audit()`: `accountId` is optional (some events are user-level, not account-level)

---

## 10. Failure modes

| Failure | Handling | Caller Impact |
|---|---|---|
| PB write fails (network/5xx) | Retry 3x with exponential backoff (100ms, 500ms, 2s) | Caller gets no error; entry is buffered; sync on reconnect |
| PB persistently unreachable | Logger mode = `buffering`; writes go to in-memory buffer | Caller unaffected; eventually if buffer fills, oldest entries drop |
| Buffer overflow (10K entries) | Drop oldest, emit a CRITICAL alert about the drop | Caller unaffected |
| Invalid input (bad LogType, empty message) | Throw `\InvalidArgumentException` | Caller gets exception, must fix |
| Auth boundary violation (audit without userId) | Throw `AuthBoundaryViolation` | Caller must fix the call site |
| Disk full (only relevant if we add file drivers later) | N/A in v0.1 (no file driver) | N/A |

---

## 11. User simulation

These are scenarios a developer can run to verify the build matches the formation:

1. **Test: write() with Critical level persists immediately**
   - Given: Logger with PB online
   - When: `write(LogType::Critical, "system-failure", ["detail" => "..."])`
   - Then: PB `email_audit` has a row with `level=Critical` within 100ms

2. **Test: write() with Debug level goes to buffer, not PB (when below threshold)**
   - Given: Config has `level_threshold = Info`
   - When: `write(LogType::Debug, "test")`
   - Then: No PB row; entry may be in memory buffer; metric counter incremented

3. **Test: audit() requires userId**
   - Given: Logger
   - When: `audit("email-sent", ["foo" => "bar"])` (no userId)
   - Then: Throws `AuthBoundaryViolation`

4. **Test: audit() with userId writes to PB**
   - Given: Logger
   - When: `audit("email-sent", ["user_id" => "u1", "account_id" => "a1", "message_id" => "m1"])`
   - Then: PB `email_audit` has a row with `user_id=u1, action=email-sent`

5. **Test: PB offline → buffer → reconnect → sync**
   - Given: Logger, PB unreachable (mock)
   - When: `write(LogType::Info, "msg", [...])` 5 times
   - Then: 5 entries in buffer, no PB writes
   - When: PB becomes reachable
   - Then: All 5 entries are written to PB; buffer empty

6. **Test: getRecentActivity() returns the user's recent entries**
   - Given: 100 buffer entries, 30 for user u1, 70 for user u2
   - When: `getRecentActivity("u1", 10)`
   - Then: Returns 10 most recent entries for u1, in reverse chronological order

7. **Test: invalid LogType throws**
   - Given: Logger
   - When: `write("not-a-level", "msg")`
   - Then: Throws `\TypeError` (PHP enforces enum)

8. **Test: message > 8192 chars is truncated**
   - Given: Logger
   - When: `write(LogType::Info, str_repeat("x", 10000))`
   - Then: Message is truncated to 8192 chars; a WARNING log is emitted

---

## 12. Reproducibility test

A new implementer can follow these steps to build a working `log` module from this formation:

1. Create `src/Log/LogType.php` (PHP enum with 5 cases)
2. Create `src/Log/LogEntry.php` (readonly class with the 9 fields from §3)
3. Create `src/Log/Driver.php` (interface with `write` and `flush` methods)
4. Create `src/Log/PbAuditDriver.php` (concrete Driver that writes to `email_audit` via PocketBaseAdapter)
5. Create `src/Log/Logger.php` (the main class with 6 public methods, 3 in-memory fields: buffer, drivers, pbAdapter)
6. Run `composer dump-autoload`
7. Run the 8 user simulation tests (or PHPUnit equivalents)
8. Deploy to staging, run `php -l` to verify syntax, then `vendor/bin/phpunit` to verify behaviour
9. If all 8 tests pass, the formation is satisfied; the module is ready to be used by `compose`

---

## 13. Substitutability (FvW v8 §37.4)

A drop-in alternative implementation of this module MUST:

1. Provide a class in namespace `FreshCloud\Mail\Log\*` (or a fully qualified alternative) that implements the 6 public methods
2. Honour the `LogType` enum (same 5 values, same integer mappings)
3. Honour the `LogEntry` shape (9 fields, same types)
4. Write audit events to the `email_audit` PB collection (same fields, same indexes)
5. Buffer writes when PB is unreachable; sync on reconnect
6. Emit the 3 metrics (`freshcloud_mail_log_writes_total`, `freshcloud_mail_log_buffer_size`, `freshcloud_mail_log_sync_lag_seconds`)
7. Throw on `audit()` without `userId` in context
8. Truncate messages > 8192 chars with a WARNING log

**Oscar can substitute** its own logging module by:
- Writing a class with the same 6 methods
- Wiring it into the bridge (no changes to other modules needed)

**FreshCloud can update** this implementation without breaking Oscar (because Oscar's substitute is independent).

---

## 14. Invariants (the 5 things that must always hold)

1. **CRITICAL-level logs always persist immediately.** No buffering, no batching. A CRITICAL log appears in PB within 100ms.
2. **Audit logs require user context or fail fast.** Every `audit()` call MUST include `user_id` in context. No anonymous audit events.
3. **All log entries have a unique id.** UUIDv4. Within a 1ms window, no two entries from the same source share an id (collision probability is astronomically low with UUIDv4, but we verify at the assertion level).
4. **The buffer is bounded.** Maximum 10,000 entries. Overflow drops oldest with a CRITICAL alert. The buffer is never allowed to grow unbounded.
5. **One source of truth for audit logs.** All audit data is in PB `email_audit`. The buffer is a query cache, not a storage. PB is canonical.

---

## 15. Sweep fields applied (the 14 design intent fields)

These are the design intent for how this module behaves in the FreshCloud multi-user, world-class, observable, testable data plane. They show up in the PHP code as fields, method signatures, and behaviour.

### Sweep 1: Multi-user + PocketBase (6 fields)

| Field | Intent | How it shows in the PHP code |
|---|---|---|
| `user_scoping` | Every log line is associated with a user session OR an anonymous system event | `LogEntry.user_id` is nullable; non-null for user events; null for system events |
| `pb_integration` | Audit trails persist in PB `email_audit` (not files) | `PbAuditDriver` is the only driver in v0.1; uses `PocketBaseAdapter` interface; writes to `email_audit` collection |
| `auth_boundary` | Logs emitted outside an auth context (e.g. system startup) are allowed; audit events are NOT | `write()` works with no userId; `audit()` throws `AuthBoundaryViolation` if no userId |
| `state_isolation` | Log records are scoped by user_id at query time | `getRecentActivity(userId, limit)` filters by userId; PB index on `(user_id, created_at DESC)` |
| `logout_behaviour` | Session-scoped logs are NOT purged on logout (they're append-only) | `flush()` does NOT delete; retention policy (90 days) handles pruning |
| `multi_account` | Logs include `account_id` for per-account filtering (not just per-user) | `LogEntry.account_id` is nullable; non-null when the event is account-specific |

### Sweep 2: World-class Abilities (5 fields)

| Field | Intent | How it shows in the PHP code |
|---|---|---|
| `connection_pool` | N/A for logging (logs are async to PB) | Not applicable; documented as N/A in the design |
| `offline_cache` | Logs survive PB outages via in-memory buffer | 10,000-entry circular buffer in `Logger`; `flush()` writes all buffered entries to PB; PB outage → mode = `buffering` |
| `sieve_integration` | Sieve execution results are logged via this module | Future: Sieve module calls `write(LogType::Info, "sieve-executed", ["script_name" => ..., "matched_count" => ...])` |
| `audit_logging` | This module IS the audit system | `audit()` writes to `email_audit`; SHA256-hashed entry ids (to detect duplicates); immutable (no UPDATE on PB rows) |
| `ux_primitives` | Diagnostic data for support and activity feed | `getRecentActivity(userId, limit)` and `getRecentErrors(userId, since)` are the UI bind points |

### Sweep 3: Migration + Observability + Tests (3 fields)

| Field | Intent | How it shows in the PHP code |
|---|---|---|
| `migration` | Schema versioning for `email_audit` table | The PB collection has a `schema_version` field (set on collection create); future migrations use a `migrations` table |
| `observability` | Real-time metrics on log health | 3 metrics emitted: `freshcloud_mail_log_writes_total{level, status}`, `freshcloud_mail_log_buffer_size`, `freshcloud_mail_log_sync_lag_seconds` |
| `tests` | High coverage of business logic | 8 user simulation tests in §11, mapped to PHPUnit; target ≥ 90% line coverage of `src/Log/` |

---

## 16. Cross-references

- **§11 (Recipe Book shape)**: This formation is the FORMATION half of the §11 10-item recipe. The build will produce the FINALIZED half.
- **§37 (Lego blocks)**: This is a Lego block. It exports `FreshCloud\Mail\Log\Logger` as a substitutable interface. Oscar can swap it.
- **§38 (Two-Recipe Model)**: This is the FORMATION phase. Next: Plan-as-Seed, then Build, then Finalize, then Archive.
- **§19 (Recipe Extraction)**: The EXISTING 10 files in `recipes/engine/log/mailso-actual/` are the FvRE-extracted MailSo actual. They are HISTORICAL, not the spec for the build.
- **§22 (Behaviour-First Codex)**: The C1-C7 in this formation maps to the same C1-C7 the build will produce in the finalized-recipe.
- **§35 (Module-Pact Registration)**: The `module.json` will declare `constitutional_fragments: ["freshcloud.mail.log.001"]`.
- **Dependent modules**: `compose` (already uses Logger via the LoggerInterface), `auth` (future), `accounts-ui` (future), `smtp` (future), `imap-client` (future).
- **MailSo reference**: `mailso-actual/` subfolder contains the 10 files we extracted from MailSo. They are read-only historical record.

---

## 17. What we are intentionally NOT doing (deferred to future)

These are things MailSo's `log` module did, or that we MIGHT do later, but are NOT in our v0.1.0 build intent:

- **Syslog driver** — MailSo had one. We skip it in v0.1. Use PB instead.
- **Stderr driver** — MailSo had one. We skip it. PHP-FPM error_log is sufficient.
- **File driver** — MailSo had one (with daily rotation, 30-day retention). We skip it. PB is the source of truth.
- **Log shipping to external systems** (ELK, Splunk) — out of scope for v0.1.
- **Multi-region log replication** — out of scope.
- **GDPR-aware retention** (auto-delete user X's logs on user X's request) — out of scope; retention is uniform 90 days.
- **Log correlation IDs across microservices** — out of scope; we're a single PHP process.
- **Performance profiling via logs** — out of scope; use a profiler.
- **Log compression** — out of scope; PB handles storage efficiently.
- **Real-time log streaming to clients** (WebSocket) — out of scope; clients use `getRecentActivity()` polling.

**MailSo had 7 PHP files: Logger, Driver, Drivers/{File, StderrStream, Syslog}, Enumerations/Type, Inherit.** We will have **4 PHP files: LogType (enum), LogEntry (data shape), Driver (interface), Logger (main class)**, plus `PbAuditDriver` as a concrete Driver implementation. Reason: simpler, no legacy drivers, no file rotation, no syslog integration. The 4 files cover everything FreshCloud needs in v0.1. If we add file drivers later, that goes in a separate formation-recipe (per the operator's forward-only doctrine).

**Deviations from MailSo are deliberate and recorded here, not as bugs, but as design decisions that fit FreshCloud's PB-first architecture.**