UX & Product Design5.0 · 0 ratings

A/B Test Hypothesis And Variant Designer

Frames a rigorous A/B test hypothesis, designs the variant, and defines metrics, sample size logic, and decision rules.

Role-BasedStep-by-StepStructured-Output

Prompt

ROLE: You are a product designer fluent in experimentation who designs tests that produce trustworthy decisions.

CONTEXT: We want to improve [TARGET_METRIC] on [SURFACE] in [PRODUCT]. The observed problem and any data: [PROBLEM_AND_DATA]. Current baseline rate: [BASELINE]. Traffic available: [TRAFFIC].

TASK: Design a defensible A/B test.
1. Write the hypothesis in 'Because we observed [X], we believe [CHANGE] will cause [EFFECT] measured by [METRIC]' form.
2. Design the variant: exactly what changes vs. control, and why that specifically should move the metric.
3. Define the primary metric, 1-2 secondary metrics, and at least one guardrail metric to catch harm.
4. State the minimum detectable effect and the rough sample/duration needed (show the reasoning, not a precise calculation).
5. Define the decision rule up front: ship / iterate / kill thresholds, and how to avoid peeking bias.
6. List confounds and how the test design controls for them.

OUTPUT FORMAT: Sections — Hypothesis | Variant Spec | Metrics (primary/secondary/guardrail) | Sample & Duration Reasoning | Decision Rule | Risks & Confounds.

CONSTRAINTS: Exactly one primary metric. Include a guardrail metric. Define success/failure thresholds before running. State assumptions explicitly; do not fabricate statistics. Avoid testing many changes at once.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Step-by-Step

Forces explicit intermediate reasoning instead of jumping to a conclusion, which improves accuracy on hard tasks.

Learn this technique
Structured Output

Pins the response to a defined structure so it drops straight into your workflow.

Learn this technique

Recommended models

claudegpt-4ogemini

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

More in UX & Product Design