YQL 抓取整个网站/域

发布于 2024-08-15 13:08:44 字数 191 浏览 10 评论 0原文

我正在尝试从域中恢复一组链接和内容。

谷歌中的查询是

"site:www.newswebsite.com search_term"

我已经看到了一些接近的东西来实现这个工作，但我似乎无法完全在整个网站上进行搜索，然后按搜索词进行过滤。

如果没有自定义数据表，这可能吗？

原文

I'm trying to scape back a set of links and content from a domain.

The Query in google would be

"site:www.newswebsite.com search_term"

I've seen some close stuff to getting this working, but I can't seem to quite get a search working across a whole website, and then filter by the search term.

Is this possible without a custom data table?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

多彩岁月 2024-08-22 13:08:44

我最终明白了它的真相。

select title,abstract,url,date from search.web(0) where query="search_term" and sites="www.website1.com,www.website2.com,www.website3.com" | sort (field='date') | reverse()

这会搜索 3 个站点，按日期排序，最新的排在最前面。有另一种方法可以反转排序，但这似乎目前有效。我认为它在排序中降序= true (field='date',descending='true')

非常有用，即使我自己这么说。

I got to the bottom of it in the end.

select title,abstract,url,date from search.web(0) where query="search_term" and sites="www.website1.com,www.website2.com,www.website3.com" | sort (field='date') | reverse()

This searches 3 sites, orders by date, and newest first. There is an alternate way to reverse the sort, but this seems to work for now. I think it's descending=true within the sort (field='date',descending='true')

Very useful, even if I do say so myself.

回复收藏 0 原文