Google Web Optimizer（A/B 测试）为什么没有明显的赢家？
我之前问过获胜需要多长时间组合出现在 Google 的网络优化工具上，但现在我在 A/B 测试期间遇到了另一个奇怪的问题：
过去两天 Google 宣布有一个“高置信度获胜者” strong>”，有 98.5% 的机会击败原始变体 27.4%。伟大的！
我决定让它继续运行以确保绝对，但奇怪的事情发生了：今天谷歌表示他们“尚未收集足够的数据来显示任何重大结果”（如下所示）。当然，这些数字略有变化，但仍然非常高：96.6% 的机会超过原始版本22%。
那么，为什么谷歌现在不太确定呢？
怎么可能从统计上显着的“高置信度”获胜者变成没有足够的数据来计算呢？我的数字是否太小，谷歌无法绝对确定还是什么？
感谢您的任何见解！
I've previously asked how long it takes for a winning combination to appear on Google's Web Optimizer, but now I have another weird problem during an A/B test:
For the past two days has Google announced that there was a "High Confidence Winner" that had a 98.5% chance of beating the original variation by 27.4%. Great!
I decided to leave it running to make absolutely sure, but something weird happened: Today Google is saying that they "haven't collected enough data yet to show any significant results" (as shown below). Sure, the figures have changed slightly, but they're still very high: 96.6% chance of beating the original by 22%.
So, why is Google not so sure now?
How could it have gone from having a statistically significant "High Confidence" winner, to not having enough data to calculate one? Are my numbers too tiny for Google to be absolutely sure or something?
Thanks for any insights!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
对于所有统计测试，都有所谓的 p 值，它是假设正在测试的内容之间没有差异，随机获得观察到的结果。因此，当您运行测试时，您需要一个较小的 p 值，以便您对结果充满信心。
因此，GWO 的 p 值必须在 1.5% 到 3.4% 之间（我猜是 2.5%，至少在这种情况下，可能取决于组合的数量）
所以当 (100% - 击败的机会%)> p-值，那么 GWO 会说它没有收集到足够的信息，并且如果组合的 (100% - 击败%的机会) < p 值，然后找到获胜者。显然，如果这条线刚刚被跨越，那么它可以很容易地返回更多的数据。
总而言之，您不应该经常检查结果，您应该设置一个测试，然后忽略它很长一段时间，然后检查结果。
不
With all statistics tests there is what's called a p-value, which is the probablity of obtaining the observed result by random chance, assuming that there is no difference between what's being tested. So when you run a test, you want a small p-value so that you can be confident with your results.
So with GWO must have a p-value between 1.5% and 3.4% (I'm guessing it's 2.5%, atleast in this case, it might be that it depends on the number of combinations)
So when (100% - chance to beat %) > p-value, then GWO will say that it has not collected enough information, and if a combination has a (100% - chance to beat %) < p-value then a winner is found. Obviously if that line is just crossed, then it could easily go back with a little more data.
To summerize, you shouldn't be checking the results frequently, you should setup a test, then ignore it for a long while then check the results.
No