From 9e07407484c01fa4ac57ba07e47f348670540c13 Mon Sep 17 00:00:00 2001 From: Arthur Belleville Date: Fri, 15 May 2026 16:39:54 +0200 Subject: [PATCH] docs(06-02): complete cmd/worker river wiring plan - 06-02-SUMMARY.md: river wiring summary with commits, decisions, threat flag review - STATE.md: decisions, metrics, notes, SUMMARY reference added for Plan 02 - ROADMAP.md: 06-02 marked complete (2/3 plans executed) --- .planning/ROADMAP.md | 4 +- .planning/STATE.md | 11 +- .../06-background-worker/06-02-SUMMARY.md | 113 ++++++++++++++++++ 3 files changed, 123 insertions(+), 5 deletions(-) create mode 100644 .planning/phases/06-background-worker/06-02-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 02ea5b1..57592d9 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -151,13 +151,13 @@ Plans: **User-in-loop:** Approve the queue library/approach (`river` vs `asynq` vs hand-rolled `pg_notify`) and pick the proof-of-life job. -**Plans:** 1/3 plans executed +**Plans:** 2/3 plans executed Plans: **Wave 1** - [x] 06-01-PLAN.md — Wave 1: go get river + ListOrphanFiles sqlc query + internal/jobs/ package (HeartbeatWorker, OrphanCleanupWorker, SlogErrorHandler) + unit tests (WORK-01 partial, WORK-02, WORK-03, WORK-04) **Wave 2** *(blocked on Wave 1 completion)* -- [ ] 06-02-PLAN.md — Wave 2: replace cmd/worker/main.go with full river wiring (rivermigrate + Client + periodic jobs + graceful shutdown) + just worker target + README section (WORK-01, WORK-02, WORK-03) +- [x] 06-02-PLAN.md — Wave 2: replace cmd/worker/main.go with full river wiring (rivermigrate + Client + periodic jobs + graceful shutdown) + just worker target + README section (WORK-01, WORK-02, WORK-03) **Wave 3** *(blocked on Wave 2 completion)* - [ ] 06-03-PLAN.md — Wave 3: Human-verify checkpoint — run just worker, observe heartbeat logs, verify graceful shutdown diff --git a/.planning/STATE.md b/.planning/STATE.md index 783a2fb..de8519e 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,13 +3,13 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: ready_to_plan -last_updated: "2026-05-15T16:34:00.000Z" +last_updated: "2026-05-15T14:38:52.742Z" progress: total_phases: 7 completed_phases: 5 total_plans: 25 - completed_plans: 23 - percent: 92 + completed_plans: 24 + percent: 96 --- # STATE @@ -82,6 +82,8 @@ See: `.planning/PROJECT.md` (updated 2026-05-14) - **GetTaskByID before UpdateTask in reorder** — preserves title+description (T-04-08), validates task-to-tablo ownership at fetch time (T-04-10) (04-03) - **fileQuerier interface in OrphanCleanupWorker** — enables mock injection for pure unit tests without real DB; pool field retained for production (06-01) - **river deps as // indirect until Plan 02** — cmd/worker wiring in Plan 02 will promote river to direct dependency; expected Go module behavior (06-01) +- **signal.NotifyContext after rivermigrate and S3 init** — startup I/O uses context.Background(); signal context created after all I/O succeeds; prevents river.Client.Start receiving a pre-cancelled context (RESEARCH Pitfall 2) (06-02) +- **pool.Close() explicit after StopAndCancel, not via defer** — StopAndCancel fully returns before pool is closed; matches PATTERNS.md pool close ordering (06-02) ## Performance Metrics @@ -102,6 +104,7 @@ See: `.planning/PROJECT.md` (updated 2026-05-14) | Phase 04-tasks-kanban P03 | ~15min | 3 tasks | 3 files | | 06-background-worker | 01 | ~15min | 2 | 9 | | Phase 06-background-worker P01 | ~15min | 2 tasks | 9 files | +| 06-background-worker | 02 | ~10min | 2 | 3 | ## Notes @@ -137,6 +140,8 @@ See: `.planning/PROJECT.md` (updated 2026-05-14) - Commits (04-03): 2b299e2 (TaskEditHandler + TaskUpdateHandler + TaskEditFragment + Sortable.js init), 5f87d7e (TaskReorderHandler + reorder test skips removed), f6deb87 (TestTaskOrderPersists active — full suite green) - Phase 6 Plan 01 SUMMARY: `.planning/phases/06-background-worker/06-01-SUMMARY.md` - Commits (06-01): 62e5e3e (river dep + ListOrphanFiles sqlc query), a1c2828 (internal/jobs package + unit tests) +- Phase 6 Plan 02 SUMMARY: `.planning/phases/06-background-worker/06-02-SUMMARY.md` +- Commits (06-02): 6e70478 (cmd/worker full river wiring), e202ad3 (just worker target + README docs) --- *Last updated: 2026-05-15 after Phase 4 Plan 03 complete (Wave 3 — inline task edit + drag-and-drop reorder, all 9 TestTask* tests active)* diff --git a/.planning/phases/06-background-worker/06-02-SUMMARY.md b/.planning/phases/06-background-worker/06-02-SUMMARY.md new file mode 100644 index 0000000..7517d75 --- /dev/null +++ b/.planning/phases/06-background-worker/06-02-SUMMARY.md @@ -0,0 +1,113 @@ +--- +phase: "06-background-worker" +plan: "02" +subsystem: "background-worker" +tags: ["river", "rivermigrate", "periodic-jobs", "worker", "justfile", "readme"] +dependency_graph: + requires: + - "06-01: river dependency in go.mod, internal/jobs package (HeartbeatWorker, OrphanCleanupWorker, SlogErrorHandler)" + provides: + - "cmd/worker/main.go: full river wiring — rivermigrate + river.Client + periodic jobs + graceful shutdown" + - "just worker: local dev target with MinIO defaults" + - "README.md: worker local dev instructions and single-worker constraint documentation" + affects: + - "backend/cmd/worker/main.go (replaced skeleton with full implementation)" + - "backend/justfile (worker target added)" + - "backend/README.md (worker section replaced)" +tech_stack: + added: [] + patterns: + - "context.Background() for startup I/O, signal.NotifyContext AFTER all startup completes (PATTERNS.md critical ordering)" + - "rivermigrate.New(riverpgxv5.New(pool), nil).Migrate() for idempotent schema migration at startup" + - "river.Client with slog.Default() Logger, SlogErrorHandler, PeriodicJobs, MaxWorkers:10" + - "StopAndCancel(10s timeout context) + explicit pool.Close() after StopAndCancel returns" +key_files: + created: [] + modified: + - "backend/cmd/worker/main.go" + - "backend/justfile" + - "backend/README.md" +decisions: + - "signal.NotifyContext created AFTER rivermigrate and S3 init (not before) — RESEARCH Pitfall 2 + PATTERNS.md critical ordering constraint" + - "pool.Close() called explicitly after StopAndCancel returns, not via defer — PATTERNS.md pool close ordering" + - "S3 init is non-optional (exits on error) — worker cannot run orphan cleanup without S3 credentials" + - "OrphanCleanupArgs PeriodicJobOpts is nil (no RunOnStart) — first cleanup at 1 hour, not on startup" +metrics: + duration: "~10min" + completed: "2026-05-15" + tasks: 2 + files: 3 +--- + +# Phase 06 Plan 02: cmd/worker Full River Wiring Summary + +**One-liner:** cmd/worker skeleton replaced with full river runtime — rivermigrate at startup, S3 init, signal context after I/O, worker registration, river.Client with two periodic jobs (heartbeat 1min RunOnStart:true, orphan cleanup 1hr), StopAndCancel graceful shutdown; `just worker` target and README docs added. + +## Tasks Completed + +| Task | Name | Commit | Files | +|------|------|--------|-------| +| 1 | Replace cmd/worker/main.go with full river wiring | 6e70478 | backend/cmd/worker/main.go | +| 2 | Add just worker target + document worker in README.md | e202ad3 | backend/justfile, backend/README.md | + +## Outcomes + +### Task 1: Full river wiring in cmd/worker/main.go + +- `rivermigrate.New(riverpgxv5.New(pool), nil).Migrate(ctx, rivermigrate.DirectionUp, nil)` at startup before client construction +- S3 env var loading: S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY, S3_SECRET_KEY, S3_REGION (default "us-east-1"), S3_USE_PATH_STYLE +- `signal.NotifyContext` created AFTER pool connect + rivermigrate + S3 init (critical ordering constraint) +- `river.AddWorker(workers, &jobs.HeartbeatWorker{})` and `river.AddWorker(workers, jobs.NewOrphanCleanupWorker(pool, fileStore))` +- `river.NewClient` with `Logger: slog.Default()`, `MaxWorkers: 10`, `ErrorHandler: &jobs.SlogErrorHandler{}` +- Two `PeriodicJob` entries: heartbeat every 1 min (RunOnStart:true), orphan cleanup every 1 hr (nil opts) +- `riverClient.Start(sigCtx)` then `<-sigCtx.Done()` then `riverClient.StopAndCancel(10s timeout)` +- `pool.Close()` after StopAndCancel returns (not via defer) +- `go build ./cmd/worker/...` and `go build ./...` both exit 0 + +### Task 2: justfile + README + +- `worker: db-up` target with all S3 MinIO dev defaults on separate continuation lines +- README "Running the Worker" section replaces the outdated Phase 1 skeleton section +- Single-worker constraint documented (D-08) +- Expected log sequence documented (worker ready, worker heartbeat, graceful shutdown messages) +- Failed job retry observation documented (SlogErrorHandler fields, 25-attempt max, attempts^4 backoff) + +## Verification Results + +``` +go build ./... → exit 0 +go build ./cmd/worker/... → exit 0 +grep "rivermigrate.New" cmd/worker/main.go → line found +grep "^worker:" justfile → 1 match +just --list | grep worker → target registered +grep "just worker" README.md → 1+ matches +``` + +## Deviations from Plan + +None — plan executed exactly as written. + +The research code example in 06-RESEARCH.md used `defer pool.Close()` in the sample, but the plan's task action explicitly specified explicit `pool.Close()` after StopAndCancel per PATTERNS.md. The implementation follows the plan (explicit close, not defer). + +## Known Stubs + +None. All river wiring is fully implemented. The worker binary will connect to Postgres, run rivermigrate, initialize S3, register workers, and process jobs when run with valid env vars. + +## Threat Flags + +None. No new network endpoints or trust-boundary crossings beyond what the plan's threat model covers (T-06-05 through T-06-08 are all addressed by the implementation). + +- T-06-05 (rivermigrate concurrent execution): README single-worker constraint documented +- T-06-06 (Start with cancelled context): signal.NotifyContext created after startup I/O +- T-06-07 (S3 credentials in justfile): dev-only MinIO defaults; production uses real env vars +- T-06-08 (Orphan cleanup wrong S3 key): uses s3_key from tablo_files row directly + +## Self-Check: PASSED + +- `backend/cmd/worker/main.go` — FOUND +- `backend/justfile` worker target — FOUND (`grep "^worker:" justfile` returns line) +- `backend/README.md` worker section — FOUND (`grep "just worker" README.md` returns line) +- commit `6e70478` — FOUND +- commit `e202ad3` — FOUND +- `go build ./...` — PASSES +- `go build ./cmd/worker/...` — PASSES