在golang中使用colly抓取一个简单的网站不会返回任何数据
我正在尝试抓取一个如下所示的简单网站：
<html>
<head>
</head>
<body>
<pre>
"Name Surname 1
Name Surname 2
Name Surname 3
Name Surname 4"
</pre>
</body>
</html>
编写了一个简单的 go 代码：
package main
import (
"fmt"
"github.com/gocolly/colly"
)
func main() {
c := colly.NewCollector(
colly.AllowedDomains("thewebsite.com"),
)
c.OnHTML("body", func(e *colly.HTMLElement) {
fmt.Println(e.Text)
})
c.OnResponse(func(r *colly.Response) {
fmt.Println(r.StatusCode)
})
c.OnRequest(func(r *colly.Request) {
fmt.Println("Visiting", r.URL)
})
c.Visit("http://thewebsite.com")
}
当我运行此代码时，我得到以下输出：
Visiting http://thewebsite.com
200
所以一切正常。该网站已成功打开，但我没有从中获取任何数据。
我尝试将 c.OnHTML 更改为 pre、body.pre - 但它们都没有按照我的预期工作。
我在这里缺少什么？
I'm trying to scrape a simple website that looks like this:
<html>
<head>
</head>
<body>
<pre>
"Name Surname 1
Name Surname 2
Name Surname 3
Name Surname 4"
</pre>
</body>
</html>
Wrote a simple go code:
package main
import (
"fmt"
"github.com/gocolly/colly"
)
func main() {
c := colly.NewCollector(
colly.AllowedDomains("thewebsite.com"),
)
c.OnHTML("body", func(e *colly.HTMLElement) {
fmt.Println(e.Text)
})
c.OnResponse(func(r *colly.Response) {
fmt.Println(r.StatusCode)
})
c.OnRequest(func(r *colly.Request) {
fmt.Println("Visiting", r.URL)
})
c.Visit("http://thewebsite.com")
}
When I run this code, I get the below output:
Visiting http://thewebsite.com
200
So everything is OK. The website is being opened successfully, but I do not get any data from it.
I've tried to change the c.OnHTML to pre, body.pre - but none of them worked as I expected to.
What am I missing here?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我遇到了类似的问题，我必须删除域限制，尽管它看起来是正确的。换句话说，尝试注释掉
AllowedDomains()位，如下所示：I had a similar problem and I had to remove the domain restriction, despite it appearing to be correct. In other words, try commenting out the
AllowedDomains()bit, like this: