免费的谷歌词频数据库？
在本周的 Stackoverflow 播客上，Jeff 提到，2004 年他编写了一个脚本，用 110,000 个英语单词在 Google 上进行查询，并收集了一个包含每个单词的点击次数的数据库。 他们在 Stackoverflow 上使用它，例如每个问题页面右侧的“相关”列表。
由于今天用类似的脚本创建其中一个是很困难的（正如 Joel 提到的，“在 30,000 个单词的情况下，你会被敲门”），我想知道是否有人知道一个更新的、免费的数据库Google 词频（例如，自那时以来肯定发生变化的 IT 词，如 jquery、ruby、azure 等）。
On the Stackoverflow podcast this week, Jeff mentioned that in 2004 he wrote a script which queried Google with 110,000 English words and collected a database containing the number of hits for each word. They use this on Stackoverflow e.g. for the "Related" list on the right-hand side of each question page.
Since creating one of these today with a similar script would be difficult (as Joel mentioned, "at 30,000 words you get a knock at your door"), I was wondering if anyone knows of a more up-to-date, free database of Google word frequencies (e.g. for IT words which have surely changed since then such as jquery, ruby, azure, etc.).
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
快速谷歌搜索（！）会出现一些点击。 这个链接看起来很有希望：
但它并不是针对 IT 词汇。
A quick Google search(!) turns up a few hits. This link looks promising:
But it's not targeted at IT words.
也许现在回答这个问题已经晚了，但我可以向你建议不同的方式。
而不是从谷歌获取“点击数”来自己计算一些近似值。 获取大量文本页面（语料库）并计算其中每个单词的数量。
我已经通过维基百科做到了这一点。 有所有 wiki 页面的转储。 您只需要编写一个解析器来提取文本并计算单词数。 结果是超过 110K 个单词（至少 2M-3M）的列表。
如果您确实需要 Google 搜索结果中的数字，您可以获取一些单词样本并查询 Google，然后对计算值进行一些标准化以匹配 Google 值。
我希望这有帮助。
It maybe late to answer this but I can propose you different way.
Instead of getting "number of hits" from Google to compute some approximation of it by yourself. Get big collection of text pages (Corpus) and count the number of each word in it.
I have done this with the Wikipedia. There is a dump of all wiki pages. You just need to write a parser to extract text and to count words. The result is a list of more then 110K words (at least 2M-3M).
If you really need numbers in Google search result you can get some sample of words and query Google and then make some normalization of computed values to match the Google values.
I hope this helps.
根据 Google 的说法，每个 IP 每天可以发送 50,000 个查询。 我真的不认为在你的朋友之间分割它是非法的。
我在每个IP每天的查询上遇到了类似的问题，但我们通过完全不同的方法解决了它。
According to Google, you may send 50,000 queries per day per one IP. I don't really think that it is illegal to split it between your friends..
I had similar problem with queries per day per IP but we solved it by totally different approach.
您可以在朋友/同事之间拆分一个列表，并使用足够大的超时，这样每个 IP 每天的请求不会超过 50,000 个，然后合并结果。 我不确定这种方法的合法性，但谷歌人使用这种方法“敲你的门”的可能性相当低。
注：根据Skuta提供的数据编辑
You can split a list between your friends/collegues and use sufficiently large timeouts so you don't exceed 50,000 requests per day per IP, and then merging the results. I'm not sure about the legality of this approach, but the probability of having Google people "knocking at your door" using this method is pretty low.
NOTE: edited according to data provided by Skuta