Correlation And Driver Analysis On A Table
Plans a sound correlation/driver analysis, computes it in SQL or pandas, and guards against spurious conclusions.
Prompt
ROLE: You are a quantitative analyst examining what drives [TARGET_METRIC]. CONTEXT: Dataset [DATASET] with candidate driver columns [CANDIDATE_FEATURES] and target [TARGET_METRIC]. Tooling: [SQL / pandas]. Grain: [GRAIN]. Engine/env: [ENVIRONMENT]. TASK: 1. State the analytical question and whether you are after association (correlation) or estimated effect (regression), and why each candidate is plausible. 2. Compute pairwise correlations (Pearson for linear, Spearman for monotonic) between each driver and the target, with sample sizes. 3. Provide the code ([SQL] using CORR()/aggregates or [pandas] with .corr()) to produce a ranked driver table. 4. Caution against common traps: confounding, reverse causality, Simpson's paradox, multicollinearity among drivers, and outlier-driven correlations -- and suggest a check for each. 5. Recommend the next step (controlled regression, segmentation, or an experiment) to move from correlation toward cause. OUTPUT FORMAT: Question framing -> Ranked driver/correlation table -> Code -> Trap checks -> Recommended next step. CONSTRAINTS: Report sample sizes alongside coefficients; ignore correlations on tiny n. Never claim causation from correlation. Check for Simpson's paradox by re-running within key segments. Note when a non-linear relationship makes Pearson misleading.
How to use this prompt
- 1
Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.
- 2
Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.
- 3
Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.
Techniques in this prompt
Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.
Learn this techniqueAsks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.
Learn this techniqueHas the model critique its own draft against criteria, then revise — raising quality in a single pass.
Learn this techniqueRecommended models
Build on this prompt
Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.
More in Data Analysis & SQL
Translate Business Questions Into SQL
Turns a plain-English stakeholder question into a correct, well-commented SQL query against a known schema.
Optimize A Slow SQL Query
Diagnoses why a query is slow and rewrites it with targeted, explained optimizations and an index plan.
Debug A SQL Query That Returns Wrong Results
Systematically finds the logic error producing incorrect numbers and delivers a corrected, verified query.
Explain An Unfamiliar SQL Query In Plain English
Reverse-engineers a complex inherited query into a clear narrative, business meaning, and risk list.