A/B testi analizi

Deney sonuçları

ab_test tablosunda her kullanıcı A (kontrol) veya B (yeni tasarım) grubunda. Önce temel tablo:

SELECT variant,
       COUNT(*) AS kullanici,
       SUM(converted) AS donusen,
       ROUND(AVG(converted) * 100, 2) AS donusum_yuzde,
       ROUND(SUM(revenue) / COUNT(*), 2) AS kullanici_basi_gelir
FROM ab_test
GROUP BY variant;

İki oran z-testi

B'nin oranı A'dan anlamlı derecede yüksek mi?

p = (x₁ + x₂)/(n₁ + n₂), SE = √(p(1−p)(1/n₁ + 1/n₂)), z = (p₂ − p₁)/SE. |z| > 1.96 ise %95 güvenle fark anlamlıdır.

WITH g AS (
  SELECT variant, COUNT(*) AS n, SUM(converted) AS x, AVG(converted) AS p
  FROM ab_test GROUP BY variant
),
a AS (SELECT * FROM g WHERE variant = 'A'),
b AS (SELECT * FROM g WHERE variant = 'B')
SELECT ROUND(a.p * 100, 2) AS p_a, ROUND(b.p * 100, 2) AS p_b,
       ROUND(100 * (b.p / a.p - 1), 1) AS lift_yuzde,
       ROUND((b.p - a.p) / SQRT(((a.x + b.x) * 1.0 / (a.n + b.n)) * (1 - (a.x + b.x) * 1.0 / (a.n + b.n)) * (1.0 / a.n + 1.0 / b.n)), 2) AS z
FROM a, b;