docs(06): create gap-closure plan 04 — list-failed-jobs CLI and web enqueue wiring
Closes SC-3 (WORK-04) with a list-failed-jobs subcommand in cmd/worker and SC-4 (WORK-01) by wiring a river insert-only client into cmd/web with a /debug/enqueue-test handler that proves the web→worker enqueue path. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Arthur Belleville
parent
7a54755618
commit
8b64b48490
1 changed files with 281 additions and 0 deletions
281
.planning/phases/06-background-worker/06-04-PLAN.md
Normal file
281
.planning/phases/06-background-worker/06-04-PLAN.md
Normal file
|
|
@ -0,0 +1,281 @@
|
|||
---
|
||||
phase: 06-background-worker
|
||||
plan: 04
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: [06-03]
|
||||
files_modified:
|
||||
- backend/cmd/worker/main.go
|
||||
- backend/internal/web/router.go
|
||||
- backend/internal/web/handlers.go
|
||||
- backend/cmd/web/main.go
|
||||
autonomous: true
|
||||
requirements: [WORK-01, WORK-04]
|
||||
gap_closure: true
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Running `go run ./cmd/worker list-failed-jobs` (with DATABASE_URL set) prints discarded river jobs from the river_job table (or empty table output if none exist)"
|
||||
- "`go run ./cmd/worker` (no subcommand) continues to start the full worker runtime as before — subcommand dispatch does not break the default path"
|
||||
- "GET /debug/enqueue-test (authenticated) inserts a HeartbeatArgs job and returns 200 with plain-text confirmation; the worker picks it up within 60 seconds"
|
||||
- "cmd/web builds without error after adding the river insert-only client"
|
||||
- "`go build ./...` exits 0 with all changes applied"
|
||||
artifacts:
|
||||
- path: "backend/cmd/worker/main.go"
|
||||
provides: "list-failed-jobs subcommand branching on os.Args[1]"
|
||||
contains: "list-failed-jobs"
|
||||
- path: "backend/internal/web/handlers.go"
|
||||
provides: "EnqueueTestHandler that inserts HeartbeatArgs via river.Client"
|
||||
exports: ["EnqueueTestHandler"]
|
||||
- path: "backend/internal/web/router.go"
|
||||
provides: "/debug/enqueue-test route wired to EnqueueTestHandler"
|
||||
contains: "/debug/enqueue-test"
|
||||
- path: "backend/cmd/web/main.go"
|
||||
provides: "river.Client (insert-only) constructed from pgxpool and passed into router deps"
|
||||
contains: "river.NewClient"
|
||||
key_links:
|
||||
- from: "backend/cmd/worker/main.go"
|
||||
to: "river_job table"
|
||||
via: "pgx query SELECT id, kind, state, attempt, max_attempts, errors, finalized_at FROM river_job WHERE state = 'discarded'"
|
||||
pattern: "list-failed-jobs"
|
||||
- from: "backend/cmd/web/main.go"
|
||||
to: "backend/internal/web/handlers.go"
|
||||
via: "river.Client passed through DebugDeps struct to EnqueueTestHandler"
|
||||
pattern: "river\\.Client"
|
||||
- from: "backend/internal/web/handlers.go"
|
||||
to: "river_job table"
|
||||
via: "riverClient.Insert(ctx, jobs.HeartbeatArgs{}, nil)"
|
||||
pattern: "Insert.*HeartbeatArgs"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Close two Phase 6 roadmap success-criteria gaps that Phase 6 plans 01-03 did not implement:
|
||||
|
||||
1. SC-3 (WORK-04): Add a `list-failed-jobs` subcommand to `cmd/worker` that queries `river_job WHERE state = 'discarded'` and prints results to stdout. This provides the "simple CLI surface" the ROADMAP success criterion requires.
|
||||
|
||||
2. SC-4 (WORK-01): Wire an insert-only `river.Client` into `cmd/web` and add one debug HTTP handler (`GET /debug/enqueue-test`) that enqueues a `HeartbeatArgs` job. This proves the web→worker job dispatch path end-to-end.
|
||||
|
||||
Purpose: Both gaps represent unproven integration contracts — the failed-job surface was only observable via logs, and the web-side enqueue path was never wired at all.
|
||||
|
||||
Output:
|
||||
- Modified `backend/cmd/worker/main.go` with subcommand dispatch
|
||||
- New `EnqueueTestHandler` in `backend/internal/web/handlers.go`
|
||||
- Modified `backend/internal/web/router.go` with `/debug/enqueue-test` route
|
||||
- Modified `backend/cmd/web/main.go` with river client construction and DebugDeps wiring
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.planning/PROJECT.md
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.planning/ROADMAP.md
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.planning/phases/06-background-worker/06-CONTEXT.md
|
||||
@/Users/arthur.belleville/Documents/perso/projects/xtablo-source/.planning/phases/06-background-worker/06-VERIFICATION.md
|
||||
|
||||
<interfaces>
|
||||
<!-- Key types the executor needs. Extracted from codebase. -->
|
||||
|
||||
From backend/internal/jobs/heartbeat.go (verified in VERIFICATION.md):
|
||||
HeartbeatArgs implements river.JobArgs with Kind() string = "heartbeat"
|
||||
|
||||
From backend/internal/web/router.go:
|
||||
func NewRouter(pinger Pinger, staticDir string, deps AuthDeps, tabloDeps TablosDeps,
|
||||
taskDeps TasksDeps, fileDeps FilesDeps, csrfKey []byte, env string,
|
||||
trustedOrigins ...string) http.Handler
|
||||
|
||||
From backend/cmd/web/main.go current signature (line 116):
|
||||
router := web.NewRouter(pool, "./static", deps, tabloDeps, taskDeps, fileDeps, csrfKey, env)
|
||||
|
||||
From backend/go.mod (lines 42-46) — river is already an indirect dependency:
|
||||
github.com/riverqueue/river v0.37.0 // indirect
|
||||
github.com/riverqueue/river/riverdriver/riverpgxv5 v0.37.0 // indirect
|
||||
|
||||
From backend/cmd/worker/main.go — river.NewClient call for reference:
|
||||
riverClient, err := river.NewClient(riverpgxv5.New(pool), &river.Config{
|
||||
Logger: slog.Default(),
|
||||
Workers: workers,
|
||||
Queues: map[string]river.QueueConfig{river.QueueDefault: {MaxWorkers: 10}},
|
||||
...
|
||||
})
|
||||
// Insert-only client: pass &river.Config{} with NO Workers field (nil workers = insert-only mode)
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add list-failed-jobs subcommand to cmd/worker</name>
|
||||
<files>backend/cmd/worker/main.go</files>
|
||||
<read_first>
|
||||
- backend/cmd/worker/main.go (full file — understand current main() structure before touching)
|
||||
- backend/go.mod (confirm river and pgx imports available)
|
||||
</read_first>
|
||||
<action>
|
||||
Modify backend/cmd/worker/main.go to dispatch on os.Args[1] before the full worker startup sequence. The subcommand check must be the FIRST thing in main() after setting up the logger — before pool creation, before river migrations, before the signal context.
|
||||
|
||||
Concrete dispatch logic: if len(os.Args) > 1 and os.Args[1] == "list-failed-jobs", run a separate listFailedJobs(dsn string) function and os.Exit(0). All other argument patterns (including no arguments) fall through to the existing worker startup sequence unchanged.
|
||||
|
||||
The listFailedJobs function must:
|
||||
1. Open a pgxpool using db.NewPool(context.Background(), dsn). If dsn is empty, log "DATABASE_URL is required but unset" and os.Exit(1).
|
||||
2. Execute the raw query: SELECT id, kind, state, attempt, max_attempts, errors, finalized_at FROM river_job WHERE state = 'discarded' ORDER BY finalized_at DESC LIMIT 100 — use pool.Query(ctx, query) directly (no sqlc generation needed for this one-off).
|
||||
3. For each row, print one line to stdout in the format: id=<uuid> kind=<kind> attempt=<n>/<max> finalized_at=<RFC3339>. Use fmt.Printf.
|
||||
4. If the query returns zero rows, print "no discarded jobs found" and exit 0.
|
||||
5. Close the pool before returning.
|
||||
|
||||
Import requirements: the pgx rows.Scan must capture id (pgx scans uuid.UUID or [16]byte), kind (string), state (string), attempt (int), max_attempts (int), errors ([]byte or pgtype.Text — skip/ignore if scan is complex, leave as nil), finalized_at (pgtype.Timestamptz or time.Time pointer). Use pgx.CollectRows or rows.Next loop — match the pattern already used in backend/internal/db/sqlc/ for pgx v5 row scanning.
|
||||
|
||||
Do NOT restructure the existing main() startup sequence. The subcommand branch is a pure prefix check; if the branch is not taken, execution falls through to the existing code verbatim.
|
||||
|
||||
After adding the listFailedJobs function and the dispatch, run `cd backend && go build ./cmd/worker` to confirm it compiles. Fix any import or type errors before declaring done.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend && go build ./cmd/worker && echo "build ok"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `go build ./cmd/worker` exits 0 with the modified file
|
||||
- `grep "list-failed-jobs" backend/cmd/worker/main.go` returns at least one match
|
||||
- `grep "listFailedJobs" backend/cmd/worker/main.go` returns at least two matches (declaration + call site)
|
||||
- `grep "os.Args" backend/cmd/worker/main.go` returns the dispatch check
|
||||
- The dispatch check appears before any pool creation code — grep line numbers must show os.Args check before `db.NewPool`
|
||||
- Running `go run ./cmd/worker list-failed-jobs` with DATABASE_URL unset prints "DATABASE_URL is required but unset" and exits non-zero (the early exit guard applies inside listFailedJobs too)
|
||||
</acceptance_criteria>
|
||||
<done>cmd/worker compiles, contains a list-failed-jobs subcommand that queries river_job WHERE state = 'discarded', and the default startup path is unchanged</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Wire river insert-only client into cmd/web and add /debug/enqueue-test handler</name>
|
||||
<files>
|
||||
backend/internal/web/handlers.go
|
||||
backend/internal/web/router.go
|
||||
backend/cmd/web/main.go
|
||||
</files>
|
||||
<read_first>
|
||||
- backend/cmd/web/main.go (full file — understand existing deps wiring before adding river)
|
||||
- backend/internal/web/router.go (full file — understand NewRouter signature and route groups)
|
||||
- backend/internal/web/handlers.go (first 50 lines — understand handler pattern to match)
|
||||
- backend/internal/jobs/heartbeat.go (confirm HeartbeatArgs type and import path)
|
||||
</read_first>
|
||||
<action>
|
||||
Three coordinated changes. Apply them in order: handlers first, then router, then cmd/web.
|
||||
|
||||
STEP A — backend/internal/web/handlers.go:
|
||||
Add a new DebugDeps struct and EnqueueTestHandler at the bottom of the file. DebugDeps has one field: RiverClient of type RiverInserter, where RiverInserter is a local interface defined in handlers.go:
|
||||
|
||||
type RiverInserter interface {
|
||||
Insert(ctx context.Context, args river.JobArgs, opts *river.InsertOpts) (*river.InsertResult, error)
|
||||
}
|
||||
|
||||
The interface approach means the handlers package does not need to import riverqueue packages directly in the non-test path — only cmd/web imports river for construction.
|
||||
|
||||
However, the river import IS needed for river.InsertOpts and river.InsertResult in the interface. Add import "github.com/riverqueue/river" to handlers.go imports.
|
||||
|
||||
Also add import "backend/internal/jobs" for jobs.HeartbeatArgs.
|
||||
|
||||
EnqueueTestHandler returns an http.HandlerFunc. It calls debugDeps.RiverClient.Insert(r.Context(), jobs.HeartbeatArgs{}, nil). On success, write HTTP 200 with plain text body "enqueued heartbeat job id=<result.Job.ID>". On error, write HTTP 500 with plain text body "enqueue failed: <err>". Do not use a template — plain w.Write() is correct here.
|
||||
|
||||
STEP B — backend/internal/web/router.go:
|
||||
Add DebugDeps as a new parameter to NewRouter after fileDeps: `debugDeps DebugDeps`. Inside the router, add the route OUTSIDE the RequireAuth group (so it is accessible for integration testing without a session cookie — the handler is debug-only, not a production surface):
|
||||
|
||||
r.Get("/debug/enqueue-test", EnqueueTestHandler(debugDeps))
|
||||
|
||||
This route goes before the final healthz/demo/static block, after the RequireAuth group closing brace.
|
||||
|
||||
STEP C — backend/cmd/web/main.go:
|
||||
Construct a river insert-only client after the pool is created but BEFORE the signal context (follow the same startup ordering as cmd/worker — river client construction is startup I/O). An insert-only client uses river.NewClient with a Config that has NO Workers field set (nil workers is the insert-only pattern per river docs):
|
||||
|
||||
riverInsertClient, err := river.NewClient(riverpgxv5.New(pool), &river.Config{})
|
||||
if err != nil {
|
||||
slog.Error("river insert client init failed", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
Required imports to add to cmd/web/main.go:
|
||||
"github.com/riverqueue/river"
|
||||
"github.com/riverqueue/river/riverdriver/riverpgxv5"
|
||||
|
||||
Create the debugDeps variable:
|
||||
debugDeps := web.DebugDeps{RiverClient: riverInsertClient}
|
||||
|
||||
Update the NewRouter call to pass debugDeps as the new final argument (before csrfKey):
|
||||
router := web.NewRouter(pool, "./static", deps, tabloDeps, taskDeps, fileDeps, debugDeps, csrfKey, env)
|
||||
|
||||
Note: NewRouter signature after Step B is:
|
||||
NewRouter(pinger, staticDir, deps, tabloDeps, taskDeps, fileDeps, debugDeps, csrfKey, env, ...trustedOrigins)
|
||||
|
||||
After all three changes, run `cd backend && go build ./...` to confirm the full module builds. Fix any import, type, or signature mismatches before declaring done.
|
||||
|
||||
The river packages are already present in go.mod as indirect dependencies (lines 42-46). Promoting them to direct imports will cause `go mod tidy` to update the indirect comment — that is expected and acceptable. Run `cd backend && go mod tidy` after the build succeeds.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend && go build ./... && echo "full build ok"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `go build ./...` exits 0 with all three files modified
|
||||
- `grep "EnqueueTestHandler" backend/internal/web/handlers.go` returns a function declaration
|
||||
- `grep "RiverInserter" backend/internal/web/handlers.go` returns the interface definition
|
||||
- `grep "/debug/enqueue-test" backend/internal/web/router.go` returns the route registration
|
||||
- `grep "river.NewClient" backend/cmd/web/main.go` returns the insert-only client construction
|
||||
- `grep "riverpgxv5" backend/cmd/web/main.go` returns the import usage
|
||||
- `grep "debugDeps" backend/cmd/web/main.go` returns at least two lines (construction + NewRouter call)
|
||||
- `grep -r "river" backend/internal/web/handlers.go` returns imports and usage lines
|
||||
- `go test ./internal/web/... -count=1` exits 0 — existing tests must not regress
|
||||
</acceptance_criteria>
|
||||
<done>cmd/web builds with river insert client; GET /debug/enqueue-test is registered; the web→worker enqueue path is provably wired</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| HTTP → EnqueueTestHandler | Unauthenticated HTTP request triggers job insertion into the database |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-06-04-01 | Elevation of Privilege | GET /debug/enqueue-test | accept | Debug-only endpoint; inserts only HeartbeatArgs which has no side effects beyond a log line. No user data accessed. Route is explicitly scoped to development use — a future hardening phase can gate it behind RequireAuth or remove it. |
|
||||
| T-06-04-02 | Denial of Service | GET /debug/enqueue-test | accept | Flooding the endpoint would insert many heartbeat jobs. River's queue has MaxWorkers:10 and heartbeat Work() is a no-op log call. No meaningful amplification. Acceptable risk for a debug route. |
|
||||
| T-06-04-03 | Information Disclosure | list-failed-jobs CLI output | mitigate | The CLI runs server-side only (not an HTTP surface). Output goes to stdout of an operator-controlled terminal. river_job errors column may contain job argument values — confirm HeartbeatArgs has no PII (it has no fields). OrphanCleanupArgs similarly has no user-identifying fields. Risk is low but operator should be aware output is unredacted. |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
After both tasks complete, run the following to confirm both gaps are closed:
|
||||
|
||||
```bash
|
||||
# 1. Full module builds
|
||||
cd /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend && go build ./... && echo "BUILD OK"
|
||||
|
||||
# 2. All existing tests still pass
|
||||
cd /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend && go test ./internal/jobs/... ./internal/web/... -count=1 -v 2>&1 | tail -20
|
||||
|
||||
# 3. SC-3 gap: list-failed-jobs subcommand exists
|
||||
grep -n "list-failed-jobs" /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend/cmd/worker/main.go
|
||||
|
||||
# 4. SC-4 gap: river wired into cmd/web
|
||||
grep -n "river" /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend/cmd/web/main.go
|
||||
|
||||
# 5. SC-4 gap: debug route registered
|
||||
grep -n "/debug/enqueue-test" /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend/internal/web/router.go
|
||||
|
||||
# 6. go mod tidy does not fail
|
||||
cd /Users/arthur.belleville/Documents/perso/projects/xtablo-source/backend && go mod tidy && echo "MOD TIDY OK"
|
||||
```
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- SC-3 closed: `go run ./cmd/worker list-failed-jobs` is a valid invocation that queries river_job for discarded jobs and prints results (or "no discarded jobs found"). The default `go run ./cmd/worker` startup path is unaffected.
|
||||
- SC-4 closed: cmd/web constructs a river insert-only client, passes it through DebugDeps, and registers GET /debug/enqueue-test which calls riverClient.Insert(ctx, HeartbeatArgs{}, nil).
|
||||
- `go build ./...` exits 0 for the entire backend module.
|
||||
- `go test ./internal/jobs/... ./internal/web/... -count=1` exits 0 — no regressions.
|
||||
- WORK-01 and WORK-04 requirements are now fully satisfied.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/06-background-worker/06-background-worker-04-SUMMARY.md`
|
||||
</output>
|
||||
Loading…
Reference in a new issue