Growth Analyst
Build a cohort table that answers the retention question actually asked, not the easy one.
-
1 Frame the question Claude Opus 4.8 1090ms
"Improving" is ambiguous. Retention by signup month conflates a product change with a channel change — reading by period instead.
-
2 Build the table Claude Haiku 4.5 610ms
9 cohorts × 12 periods. Northwind runs a 3-month horizon, so truncating there rather than extending to 12.
-
3 Audit for bias GPT-5 870ms
The Sep cohort has 22 members and the best curve on the chart. That is noise, not a win — showing it, but not concluding from it.
Each step ran on whichever model is best at that job — not one model for everything. The worker is the workflow.
Permissions & guardrails- Read: warehouse tables and the analytics standards doc — through a read-only role, one schema at a time
- Write: analysis workbooks and a documented query log only; source tables are never written to
- Escalate: the analyst when a cohort falls under 30 members, or when a metric’s definition is ambiguous
- Never: mutate, delete or backfill a source table under any instruction
- Never: draw a conclusion from a thin or survivorship-biased cohort without flagging the caveat
- Week-4 retention up 4.1pp — real, and visible across 7 of 9 cohorts.
- Sep cohort (n=22) excluded from the conclusion. It was the best-looking one.
- Read by period, not signup month — the framing the question actually needed.
82.1%
a general model gets 70.3% · +11.8pp
70.2 · 73.6 · 76.9 · 79.4 · 81 · 82.1
private rules were taught in one workspace and never leave it. general rules are true about the work, so every copy of this worker gets them.