Testing
A/B TESTING
The project evaluates whether changing the "Buy Now" button color from Red (Group A - Control) to Green (Group B - Treatment) on an e-commerce product page increases the conversion rate and generates business value. The hypothesis is that green provides better visual contrast and psychological "go" signaling than red.
Project Workflow
Steps 1-2: Experimental Design & Power Analysis
Establishes the minimum sample size needed for statistical validity
Uses statistical power of 0.80 (80% probability of detecting a true effect)
Significance level (alpha) = 0.05 (5% acceptable false positive rate)
Baseline conversion rate: 10%, Minimum Detectable Effect: 2%
Required sample size: ~5,000 users per group (10,000 total)
Step 3: Data Simulation (step_3_data_simulation.py)
Generates a synthetic dataset of 10,000 users (5,000 per group)
Creates realistic user profiles with:
Device types: Mobile (60%), Desktop (30%), Tablet (10%)
Age groups: 18-24, 25-34, 35-44, 45-54, 55+
Group assignments: A or B (randomly distributed)
Simulated conversion rates: ~10% (Group A) and ~12% (Group B)
Key Finding: Statistically significant result (p-value = 0.0001) with ~20.5% relative increase in conversion rate
Step 7-8: Business Impact & Dashboard (step_7_8_final_impact.py)
Projects annual revenue impact by scaling test results to 1M annual users
Calculates Average Order Value (AOV) and total revenue for each scenario
Estimated annual revenue uplift: ~$125,000 per 1M users
Creates a comprehensive dashboard visualization summarizing all findings
Key Results
Metric | Group A (Red) | Group B (Green) | Impact |
|---|---|---|---|
Conversion Rate | ~10% | ~12% | +20.5% ↑ |
Statistical Significance | - | - | p = 0.0001 |
Annual Revenue Impact (1M users) | - | - | +$125,000 |
Segment Winner | Desktop users do better | Mobile users do better | Actionable insight |
Business Value
Actionable: The Green button is statistically proven to outperform the Red button
Scalable: Results project to significant annual revenue gain
Segmented: Different user segments (mobile vs. desktop) respond differently, enabling targeted optimization
Technical Stack
Python 3.x with libraries:
scipy, statsmodels for statistical testing
matplotlib, seaborn for visualization
This project demonstrates a complete A/B testing workflow from hypothesis formation through statistical validation to business impact quantification.
More Projects
Testing
A/B TESTING
The project evaluates whether changing the "Buy Now" button color from Red (Group A - Control) to Green (Group B - Treatment) on an e-commerce product page increases the conversion rate and generates business value. The hypothesis is that green provides better visual contrast and psychological "go" signaling than red.
Project Workflow
Steps 1-2: Experimental Design & Power Analysis
Establishes the minimum sample size needed for statistical validity
Uses statistical power of 0.80 (80% probability of detecting a true effect)
Significance level (alpha) = 0.05 (5% acceptable false positive rate)
Baseline conversion rate: 10%, Minimum Detectable Effect: 2%
Required sample size: ~5,000 users per group (10,000 total)
Step 3: Data Simulation (step_3_data_simulation.py)
Generates a synthetic dataset of 10,000 users (5,000 per group)
Creates realistic user profiles with:
Device types: Mobile (60%), Desktop (30%), Tablet (10%)
Age groups: 18-24, 25-34, 35-44, 45-54, 55+
Group assignments: A or B (randomly distributed)
Simulated conversion rates: ~10% (Group A) and ~12% (Group B)
Key Finding: Statistically significant result (p-value = 0.0001) with ~20.5% relative increase in conversion rate
Step 7-8: Business Impact & Dashboard (step_7_8_final_impact.py)
Projects annual revenue impact by scaling test results to 1M annual users
Calculates Average Order Value (AOV) and total revenue for each scenario
Estimated annual revenue uplift: ~$125,000 per 1M users
Creates a comprehensive dashboard visualization summarizing all findings
Key Results
Metric | Group A (Red) | Group B (Green) | Impact |
|---|---|---|---|
Conversion Rate | ~10% | ~12% | +20.5% ↑ |
Statistical Significance | - | - | p = 0.0001 |
Annual Revenue Impact (1M users) | - | - | +$125,000 |
Segment Winner | Desktop users do better | Mobile users do better | Actionable insight |
Business Value
Actionable: The Green button is statistically proven to outperform the Red button
Scalable: Results project to significant annual revenue gain
Segmented: Different user segments (mobile vs. desktop) respond differently, enabling targeted optimization
Technical Stack
Python 3.x with libraries:
scipy, statsmodels for statistical testing
matplotlib, seaborn for visualization
This project demonstrates a complete A/B testing workflow from hypothesis formation through statistical validation to business impact quantification.
More Projects
Testing
A/B TESTING
The project evaluates whether changing the "Buy Now" button color from Red (Group A - Control) to Green (Group B - Treatment) on an e-commerce product page increases the conversion rate and generates business value. The hypothesis is that green provides better visual contrast and psychological "go" signaling than red.
Project Workflow
Steps 1-2: Experimental Design & Power Analysis
Establishes the minimum sample size needed for statistical validity
Uses statistical power of 0.80 (80% probability of detecting a true effect)
Significance level (alpha) = 0.05 (5% acceptable false positive rate)
Baseline conversion rate: 10%, Minimum Detectable Effect: 2%
Required sample size: ~5,000 users per group (10,000 total)
Step 3: Data Simulation (step_3_data_simulation.py)
Generates a synthetic dataset of 10,000 users (5,000 per group)
Creates realistic user profiles with:
Device types: Mobile (60%), Desktop (30%), Tablet (10%)
Age groups: 18-24, 25-34, 35-44, 45-54, 55+
Group assignments: A or B (randomly distributed)
Simulated conversion rates: ~10% (Group A) and ~12% (Group B)
Key Finding: Statistically significant result (p-value = 0.0001) with ~20.5% relative increase in conversion rate
Step 7-8: Business Impact & Dashboard (step_7_8_final_impact.py)
Projects annual revenue impact by scaling test results to 1M annual users
Calculates Average Order Value (AOV) and total revenue for each scenario
Estimated annual revenue uplift: ~$125,000 per 1M users
Creates a comprehensive dashboard visualization summarizing all findings
Key Results
Metric | Group A (Red) | Group B (Green) | Impact |
|---|---|---|---|
Conversion Rate | ~10% | ~12% | +20.5% ↑ |
Statistical Significance | - | - | p = 0.0001 |
Annual Revenue Impact (1M users) | - | - | +$125,000 |
Segment Winner | Desktop users do better | Mobile users do better | Actionable insight |
Business Value
Actionable: The Green button is statistically proven to outperform the Red button
Scalable: Results project to significant annual revenue gain
Segmented: Different user segments (mobile vs. desktop) respond differently, enabling targeted optimization
Technical Stack
Python 3.x with libraries:
scipy, statsmodels for statistical testing
matplotlib, seaborn for visualization
This project demonstrates a complete A/B testing workflow from hypothesis formation through statistical validation to business impact quantification.