# Probability Analysis: Current vs New Landing Page ## Executive Summary Current historical conversion rate: 4%. New landing-page pilot: 6 conversions from 100 visitors. The pilot is encouraging, but it is not strong enough to conclude with high confidence that the true conversion rate has permanently increased from 4% to 6%. The observed difference is: +2 percentage points. Relative improvement: 50%. However: the pilot sample contains only 100 visitors and 6 conversions. That makes the estimated 6% rate statistically unstable. Current recommendation: CONTINUE THE TEST. Do not yet treat 6% as the established long-run conversion rate. Confidence in this recommendation: HIGH. --- # 1. Baseline Visitors per month: 500. Current conversion rate: 4%. Expected monthly orders: 500 × 0.04 = 20 orders. Average order value: $30. Expected monthly revenue: 20 × $30 = $600. Baseline expected revenue: approximately $600/month. --- # 2. Pilot Result Pilot visitors: 100. Conversions: 6. Observed conversion rate: 6 / 100 = 6%. If 6% were the true future rate: 500 × 0.06 = 30 expected monthly orders. Expected revenue: 30 × $30 = $900/month. Potential increase: $300/month. But this is: a scenario not: a proven result. --- # 3. Observed Improvement Current: 4%. Pilot: 6%. Absolute improvement: 6% - 4% = 2 percentage points. Relative improvement: 2 / 4 = 50%. This sounds large. However: relative improvement does not measure: statistical certainty. --- # 4. Sample-Size Problem Only: 6 conversions were observed on the new page. With such a small conversion count: one or two extra purchases materially change the measured rate. Examples: 5 conversions: 5%. 6 conversions: 6%. 7 conversions: 7%. A single additional conversion changes the estimate by: 1 percentage point. This illustrates: high sampling variability. Status: SMALL_SAMPLE_WARNING. --- # 5. Could a 6% Result Occur Even if the True Rate Were Still 4%? Yes. Random variation can cause a true 4% conversion process to produce: more or fewer than four conversions in a sample of 100. Therefore: observing six conversions does not automatically establish: a new 6% underlying rate. --- # 6. Probability Interpretation The correct question is not: "Did the new page convert at 6%?" It did in this sample. The more important question is: "What is the plausible true conversion rate behind this sample?" With only: 100 visitors, the uncertainty remains wide. Therefore the evidence supports: PROMISING SIGNAL rather than: CONFIRMED IMPROVEMENT. --- # 7. Expected Monthly Outcomes ## Current Page Expected conversion: 4%. Expected orders: 20. Expected revenue: $600. ## New Page if True Rate = 5% Expected orders: 25. Expected revenue: $750. ## New Page if True Rate = 6% Expected orders: 30. Expected revenue: $900. ## New Page if True Rate = 7% Expected orders: 35. Expected revenue: $1,050. This demonstrates why: estimating the true rate accurately matters economically. --- # 8. Scenario Analysis ## Pessimistic Scenario True long-run conversion rate: approximately 4%. Interpretation: The apparent 6% pilot was mostly sampling noise. Expected monthly orders: 20. Expected monthly revenue: $600. Result: little or no true improvement. --- ## Base Scenario Assume the new page produces: approximately 5-5.5%. At 5.25%: 500 × 0.0525 = 26.25 expected orders. Expected revenue: 26.25 × $30 = $787.50. Potential monthly improvement vs baseline: approximately $187.50. --- ## Optimistic Scenario True long-run conversion: approximately 6-7%. At 6.5%: 500 × 0.065 = 32.5 expected orders. Expected revenue: 32.5 × $30 = $975. Potential monthly increase: approximately $375. --- # 9. Decision Risk The primary risk is: EARLY WINNER SELECTION. If the business adopts the new page immediately because: 6% > 4%, it may be responding to: sampling noise. This is particularly important when: the observed conversion count is small. --- # 10. Expected Value of More Information Additional traffic has analytical value. Each additional observation helps distinguish: a true improvement from: random fluctuation. Because the decision is: reversible and: continued testing is relatively inexpensive, collecting more evidence is attractive. --- # 11. Recommended Test Strategy Continue: A/B testing if operationally practical. Track: visitors conversions revenue device type traffic source customer quality refund rate and: downstream conversion quality. Do not optimize only: initial purchase conversion if the new page attracts: lower-quality customers. --- # 12. Important Statistical Warning Do not repeatedly check the test every few visitors and stop: the first time the new version looks better. Repeated early stopping can increase: false-positive risk. Define: a planned evaluation rule before declaring a winner. --- # 13. What Would Strengthen the Evidence? More confidence would come from: - a larger sample - more conversions - randomized traffic allocation - similar traffic mix between variants - test duration covering normal weekly variation - stable pricing - stable campaign sources - no simultaneous major website changes - consistent order-value measurement - repeatable improvement across future samples. --- # 14. Recommendation Current recommendation: CONTINUE TESTING. Do not label: 6% as: the established future conversion rate. Use the result as: a positive signal. The new page may be better, but: evidence is currently insufficient for a high-confidence point estimate. --- # Probability Assessment Observed pilot conversion: 6%. Historical baseline: 4%. Interpretation: PROMISING BUT UNCERTAIN. Confidence that the pilot measurement itself is correct: HIGH. Confidence that true long-term conversion is exactly 6%: LOW. Confidence that more testing is warranted: HIGH. --- # Audit Event Definition: PASS. Sample Size: WARNING. Base Rate: PASS. Historical Benchmark: PASS. False Precision: AVOIDED. Causality: NOT OVERCLAIMED. Expected Value: USED APPROPRIATELY. Scenario Range: PASS. Final Status: SMALL_SAMPLE_WARNING + ESTIMATE_RANGE_RECOMMENDED + PROBABILITY_ANALYSIS_VERIFIED. # END OF EXAMPLE 1 This example is complete.