php Curl抓取搜索结果页面得问题

Question

求帮助.最近做一个电影搜索功能,需要抓取btso的搜索结果...但是用curl怎么设置都是返回403.也试过加入头信息,也不行,求各位大佬们帮帮忙!
谢谢了!

$url = 'https://btso.pw/search/'.$keyword;
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_HEADER, FALSE);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        curl_setopt($ch, CURLOPT_REFERER, "https://btso.pw/search/");
        dd(curl_exec($ch));

Answer 1

你先在浏览器中抓取自然浏览下的请求信息：

curl 'https://btso.pw/search/FSET-391' 
-H 'Connection: keep-alive' 
-H 'Pragma: no-cache' -H 'Cache-Control: no-cache' 
-H 'Upgrade-Insecure-Requests: 1' -H 'DNT: 1' 
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36' 
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' 
-H 'Referer: https://btso.pw/search/' 
-H 'Accept-Encoding: gzip, deflate, br' 
-H 'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8,zh-TW;q=0.7' 
-H 'Cookie: AD_enterTime=1540175511; AD_adca_b_SM_T_728x90=0; AD_jav_b_SM_T_728x90=0; AD_javu_b_SM_T_728x90=0; AD_wav_b_SM_T_728x90=0; AD_adst_b_SM_T_728x90=1; AD_exoc_b_SM_T_728x90=1; AD_adma_b_POPUNDER=2' 
--compressed ;

之后将这个信息转换为PHP即可：

$headers     = array(
    'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36',
    'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'Cookie: AD_enterTime=1540175511; AD_adca_b_SM_T_728x90=0; AD_jav_b_SM_T_728x90=0; AD_javu_b_SM_T_728x90=0; AD_wav_b_SM_T_728x90=0; AD_adst_b_SM_T_728x90=1; AD_exoc_b_SM_T_728x90=1; AD_adma_b_POPUNDER=2',
    'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8,zh-TW;q=0.7',
    'Upgrade-Insecure-Requests: 1',
    'Connection: keep-alive',
    'Referer: https://btso.pw/search/'
);
$url         = 'https://btso.pw/search/FSET-391';
$ch          = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
var_dump(curl_exec($ch));

$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
var_dump($httpCode);