2

I'm running an a/b test on an advertisement and I'd like to test for statistical significance. How would I design a test and prove whether or not conversions/clicks are statistically different from test A and test B?

Test A: 1000 clicks, 900 conversions

Test B: 380 clicks, 320 conversions

Thanks for your help guys.

Tony
  • 21

1 Answers1

0

I had a different answer here earlier, but now I'm pretty sure it was wrong. There are numerous online calculators for determining whether or not results like these are statistically significant. They all seem to use a form of Chi squared test.

So the answer to your question is: Don't design a new test for this purpose. Use what appears to be the standard test (Chi squared).

Aaron Golden
  • 1,031
  • 6
  • 12
  • P values are usually treated under roughly these cutoffs: .1-.08 slight significant evidence(S.E) against the null hypothesis(N.H), .08-.05 some S.E against the N.H, .05-.02 strong S.E against the N.H, <.02 Very strong S.E. against the N.H.

    I would consider this to be 'some' evidence.

     – Display Name Oct 16 '13 at 23:19
  • @DisplayName Sorry to deprive your comment of context. I'm pretty sure the p-value from my old answer was incorrect anyway since it didn't agree with any of the online calculators. – Aaron Golden Oct 17 '13 at 21:24