phpcurl数据抓取

发布于 2024-12-11 19:28:51 字数 1149 浏览 0 评论 0原文

我有一个 CURL 代码来从网站获取数据,它在过去几个月工作正常,但突然停止为我工作,它说

HTTP/1.0 302 Moved Temporarily

我的代码是:

$ch = curl_init();
curl_setopt($ch, CURLOPT_REFERER, $baseUrl);
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
//curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE         , $phpSId);
curl_setopt($ch, CURLOPT_COOKIEJAR      , $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE     , $cookie);
curl_setopt($ch, CURLOPT_USERAGENT      , "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1");
curl_setopt($ch, CURLOPT_TIMEOUT        , 40);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST , 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER , 0);
curl_setopt($ch, CURLOPT_URL            , $url);
curl_setopt($ch, CURLOPT_HEADER         , 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION , 1);
curl_setopt($ch, CURLOPT_POST           , 1);
curl_setopt($ch, CURLOPT_POSTFIELDS     , $data);
$result = curl_exec($ch);
curl_close ($ch);
unset($ch); 
die($result);

请帮忙,提前致谢

I have a CURL code to fetch data from a site it is working fine for last few months but suddenly stop working for me it says

HTTP/1.0 302 Moved Temporarily

my code is:

$ch = curl_init();
curl_setopt($ch, CURLOPT_REFERER, $baseUrl);
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
//curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE         , $phpSId);
curl_setopt($ch, CURLOPT_COOKIEJAR      , $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE     , $cookie);
curl_setopt($ch, CURLOPT_USERAGENT      , "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1");
curl_setopt($ch, CURLOPT_TIMEOUT        , 40);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST , 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER , 0);
curl_setopt($ch, CURLOPT_URL            , $url);
curl_setopt($ch, CURLOPT_HEADER         , 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION , 1);
curl_setopt($ch, CURLOPT_POST           , 1);
curl_setopt($ch, CURLOPT_POSTFIELDS     , $data);
$result = curl_exec($ch);
curl_close ($ch);
unset($ch); 
die($result);

Please help, thanks in advance

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

野稚 2024-12-18 19:28:51

指定的选项已经使curl遵循重定向。但是,在重定向链较长的情况下,您可能需要增加 CURLOPT_MAXREDIRS

您可以使用数据包转储程序(例如wireshark)来检查curl 发送了哪些请求。这可能只是被抓取网站中的一个错误,导致其无限重定向。

The specified options already make curl follow redirects. However, in the case of a long redirect chain, you may want to increase CURLOPT_MAXREDIRS.

You can use a packet dumper such as wireshark to check which requests are sent by curl. It may be simply a bug in the scraped website which causes it to redirect infinitely.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文