# Inbound Email Path Migration Scope

**Date:** 2026-05-21
**Status:** Scoping complete; ready for implementation planning
**Successor to:** Telepathy JMAP outbound (shipped in fe5896b)

---

## 1. Current State

The inbound email path is split across **two Python services**:

### Muse Inbox (`/data/muse/smtp_handler.py` — 360 lines)
- **Purpose:** SMTP server on port 25 accepting inbound mail for `muse@hydrascale.net` and `muse@cortex.hydrascale.net`
- **Architecture:** aiosmtpd + aiohttp async event loop
- **Key features:**
  - Parses MIME emails
  - Logs raw emails to `inbox.jsonl` (similar to Telepathy's `messages.jsonl`)
  - Dispatches to Symbiont API `/task` endpoint (port 8111) with task prompt + context
  - Implements exponential backoff retry (3 attempts) on transient Symbiont failures
  - Sends reply via Telepathy `/email` (JMAP)
  - Marks inbox entries as `processed=True` after success

### Telepathy (`/data/telepathy/app.py` — 143 lines; `/data/telepathy/mailer.py` — 204 lines)
- **Purpose:** HTTP API (FastAPI on port 8114) for message persistence and JMAP email sending
- **Endpoints:** `POST /messages`, `GET /messages`, `GET /messages/unread`, `POST /messages/{id}/read`, `POST /email`, `GET /health`
- **Dependencies:** requests, FastAPI, Pydantic

### Data Flow (Merged View)
```
Inbound email (SMTP:25, Muse)
  → Parse MIME + log to inbox.jsonl
  → POST /task to Symbiont (8111)
  → Claude CLI produces reply
  → POST /email to Telepathy (8114)
  → JMAP send via Fastmail
  → Mark processed in inbox.jsonl
  → (Optionally) Mark Telepathy message as read
```

---

## 2. Proposed Elixir Architecture

**Goal:** Consolidate both services into Symbiont.Telepathy supervision tree on port 8111 (single app).

### New Modules

| Module | Purpose | Est. Lines |
|--------|---------|-----------|
| `Symbiont.Telepathy.SMTPHandler` | gen_smtp_server_session callbacks | 100-150 |
| `Symbiont.Telepathy.InboxStore` | JSONL-backed inbox (like MessageStore) | 80-120 |
| `Symbiont.Telepathy.Pipeline` | Orchestrates receive → Symbiont dispatch → reply → mark read | 150-200 |
| HTTP endpoints (extend `Symbiont.API.ex`) | `/messages`, `/email`, `/health` | 80-120 |
| Tests | SMTP integration, dispatch retry, pipeline | 150-200 |

### Supervision Tree (Updated)
```
Symbiont.Telepathy.Supervisor (one_for_one)
  +-- MessageStore (JSONL: Fastmail-fetched emails)
  +-- InboxStore (JSONL: SMTP-received emails)
  +-- JMAP (GenServer: Fastmail session + sender)
  +-- SMTPHandler (wraps :gen_smtp listener on port 25)
  +-- TaskSupervisor (spawns Pipeline.process tasks)
```

### Key Design Decisions
1. **Inbox vs Messages:** Separate stores because they track different email directions (received vs polled from Fastmail). Both feed the UI and dispatch logic.
2. **Retry logic:** Implement as a simple loop in Pipeline.process (not a separate backoff supervisor) since Muse inbox already handles transient retries.
3. **API consolidation:** POST `/email` becomes a direct call to `JMAP.send_email/3` instead of an HTTP hop; remove port 8114 entirely.

---

## 3. Complexity Estimate

**Total LOC: ~650-870 across modules + tests**

| Category | Effort | Notes |
|----------|--------|-------|
| SMTP handler (gen_smtp) | Medium (100-150) | Straightforward protocol impl; risk is MIME edge cases |
| Pipeline + dispatch | Medium (150-200) | Needs retry logic, timeout handling, error reporting |
| Inbox store | Small (80-120) | Clone MessageStore pattern; simple JSONL ops |
| API endpoints | Small (80-120) | Thin wrappers around store/sender; mostly copy-paste |
| Tests + integration | Medium (150-200) | Must verify SMTP flow, Symbiont dispatch, reply send |
| **Refactor/Config** | **Small** | Update Application.ex, mix.exs (add :gen_smtp), runtime.exs |

**Blockers:**
- None identified. Both Muse Inbox and Telepathy run standalone with no inter-service locks. Can be developed/tested in parallel with Python services running.

**Risks:**
1. **Email loss on crashes:** If Elixir app restarts mid-Pipeline.process, inbox entry is not marked processed. Mitigation: restore from unprocessed entries on startup.
2. **gen_smtp limitations:** Erlang SMTP impl is less polished than Python aiosmtpd. May hit MIME parsing edge cases. Mitigation: test with real emails from inbox.jsonl.
3. **Symbiont dispatch timeout:** Task may take >600s. Current SMTP client will time out waiting for reply. Mitigation: return 250 OK immediately (like Muse does now), dispatch async.

---

## 4. Recommendation

**Start next session (after Michael confirms).**

**Rationale:**
- Work is well-scoped with a clear blueprint and existing code to port
- JMAP client already shipped; this is a natural successor
- System currently stable (both services running)
- Estimated 2-3 focused sessions including tests and cutover
- **No explicit urgency signal from Michael** (reflection logs don't mention inbound as a blocker)
- Higher-priority items remain blocked (blog project, permissions)

**Decision factors to verify with Michael:**
1. Is the "19 unread" bug urgent enough to fast-track this? (Elixir port fixes it by design)
2. Are there other blockers (blog, permissions) that should be prioritized first?
3. Can SMTP listen on a non-privileged port during development (e.g., 10025) for testing?

---

## 5. Next Steps (If Approved)

1. Confirm with Michael: urgency, priority relative to blog/permissions, port 25 availability
2. Add `:gen_smtp ~> 1.2` to `mix.exs`
3. Implement SMTPHandler as gen_smtp_server_session behaviour
4. Implement InboxStore (clone MessageStore, s/messages/inbox/)
5. Implement Pipeline with retry loop and Symbiont dispatch
6. Add HTTP endpoints to API.ex
7. Integration test: send email to muse@, verify dispatch, verify reply
8. Cutover: stop Python services, verify Elixir handles full flow
9. Decommission `/data/muse/` and `/data/telepathy/` services

---

**Appendix: Python Source Files for Reference**

- `/data/muse/smtp_handler.py` (360 lines) — aiosmtpd handler + Symbiont dispatch logic
- `/data/telepathy/app.py` (143 lines) — FastAPI message/email endpoints
- `/data/telepathy/mailer.py` (204 lines) — JMAP sender (already ported to Elixir)
- `/data/muse/inbox.jsonl` — inbox log (schema: timestamp, from, subject, message_id, body, processed)
- `/data/telepathy/messages.jsonl` — messages log (schema: id, timestamp, source, subject, content, read)