Hypothesis, metric and decision rule
Write the test down before it starts.
Lesson 17 of 32~19 min of learningIncludes ~12 min for questions and tasks
Contents1 of 37 steps
Maya
Theo
You
Alex Tutor
Alex Tutor
Question 1
Why write the plan before launch rather than after the data comes in? Select all that apply.
Alex Tutor
You
Alex Tutor
Maya
Alex Tutor
Alex Tutor
Question 2
Which of these is a testable hypothesis for the Halves paywall?
Alex Tutor
Theo
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 3
Match each role in the paywall test plan to its metric.
Pick an answer for each row. Choosing an answer that is already used moves it to this row.
- Primary metricChoose an answer
- GuardrailChoose an answer
- Secondary metricChoose an answer
- Exploratory cutChoose an answer
Alex Tutor
Maya
Alex Tutor
You
Alex Tutor
Alex Tutor
Question 4
The guardrail's difference came out “not statistically significant”. That's enough to conclude the new paywall did no harm.
Alex Tutor
Alex Tutor
Maya
Alex Tutor
You
Alex Tutor
Question 5
The test ends. Primary metric, treatment − control: +0.4 pp, 95% interval −0.5 to +1.3 pp. The guardrail passed. What does the decision rule say?
Alex Tutor
Maya
You
Alex Tutor
Question 6Short answer · AI-checked task
Write a test plan for invite-first onboarding in 4–7 lines: a hypothesis, one primary metric with its window, one or two guardrails with a margin, and a decision rule that covers ship, don't ship and inconclusive.
Write your answer and get a score with feedback from our AI reviewer.
Log in to get AI feedbackAlex Tutor
That’s the lesson. You answered every task — nicely done.