Python是否可以从邮政请求中获得请求有效载荷?

发布于 2025-01-22 11:57:11 字数 621 浏览 2 评论 0原文

我正在尝试将此URL的名称和价格进行封存 https://pricempire.com/search.com/search 请求包。当我加载站点时,有一个名为项目的帖子请求,在请求有效载荷中有一个称为Captchatoken的属性,基本上我想捕获此属性并使用它来为其他页面创建请求。我一直在使用硒来刮擦名称和价格,但是要加载每页需要20秒钟的时间非常慢。因此,我的目标是在请求有效载荷中捕获Captchatoken,看起来像这样:

json_data = {
    'page': 1,
    'priceMin': 0,
    'orderBy': 'price_desc',
    'captchaToken': 'aaabbbcccxxxzzzyyy',
    'priceMax': 200000,
    'collections': [],
    'weaponIds': [],
    'wears': [],
    'priceProvider': 'buff163',
}

加快网站的刮擦。但是我不知道是否可能,从我的研究中,我不知道这个问题的答案。

I'm trying to webscrape the names and prices for this URL https://pricempire.com/search via the requests package. When I load the site there is a POST request called items and inside the request payload there is a attribute called captchaToken, basically I want to capture this attribute and use it to create requests for the other pages. I've been using Selenium to scrape the names and prices but it is very slow having to take 20 seconds to load each page. So my goal is to capture the captchaToken in the request payload which looks like this:

json_data = {
    'page': 1,
    'priceMin': 0,
    'orderBy': 'price_desc',
    'captchaToken': 'aaabbbcccxxxzzzyyy',
    'priceMax': 200000,
    'collections': [],
    'weaponIds': [],
    'wears': [],
    'priceProvider': 'buff163',
}

to speed up scraping of the site. But I don't know if its possible and from my research I don't know the answer to this question.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

巡山小妖精 2025-01-29 11:57:11

在POST请求中,您必须添加服务器作为有效载荷作为JSON响应和CAPTCHA令牌的数据也是有效载荷数据的一部分。这是一个示例,如何从API中获取数据(名称,价格)为邮政方法。

import requests
import json
URL = "https://public-api.pricempire.com/api/search/items"

body= {"page":1,"priceMin":0,"orderBy":"price_desc","captchaToken":"03AGdBq271Msp7k_yCTzgNsheZ1yRqLWykDZL17tIK9_YAVo2uZGc3cLH0sNhuZOFsnymBSAbuzRRo2w_Cy6kEEMxaRxgkuZUlXFcDzRPWgYs-Hy-fV5SpxLjU8rACYW3KwZ8y-js1Dye8weAdMfZSPeEBgQ9YP3zdbaPrUOJAHHmjkpqTxH7vPW-Cd2PXHtZf5NlgVkxCBUKIESAyMJ6FyKdNz_WxYdIJvK4uQa6nBdHxMlmQZx6rUgus65NxZkwTaY3BO36ju68WNerv-fQBqFdIz_6jUPfav41DYFiApv9O-MbdASQqpS-ma1TG76mQ82OQdzkqqvpZtAksBGa836HzsxfaOecgbZ2YbswAHr1dXxl919DbRnZum4Wr-UUZMQ66j8Iy5UA_g4B3Ir7IxTf50KhTOrNHtqIIYuBR4Vfz6scc5c7XqATeqMoMvL-06wbBWVATSI44","priceMax":200000,"collections":[],"weaponIds":[],"wears":[],"priceProvider":"buff163"}

headers={
    'content-type': 'application/json',
    'User-Agent':'mozila/5.0/'
    }
jsonData=requests.post(URL,headers=headers,data=json.dumps(body)).json()


for item in jsonData['items']:
    name= item['name'].replace('★','').replace('|','').strip()
    price=item['price']['price']
    print(name)
    print(price)

输出:

Souvenir AWP  Dragon Lore (Minimal Wear)
100000000
Sticker  iBUYPOWER (Holo)  Katowice 2014    
49999900
Sticker  Titan (Holo)  Katowice 2014        
49999800
Souvenir AWP  Dragon Lore (Field-Tested)    
36388800
Sticker  Reason Gaming (Holo)  Katowice 2014
30000000
Souvenir AWP  Dragon Lore (Battle-Scarred)  
23618000
Sport Gloves  Pandora's Box (Factory New)
22000000
Sticker  Team LDLC.com (Holo)  Katowice 2014
16750000
StatTrak™ Ursus Knife  Crimson Web (Factory New)
15000000
StatTrak™ Talon Knife  Crimson Web (Factory New)
15000000
StatTrak™ Nomad Knife  Safari Mesh (Battle-Scarred)
15000000
Survival Knife  Crimson Web (Factory New)
14999999
StatTrak™ Stiletto Knife  Slaughter (Field-Tested)
14888800
Sport Gloves  Vice (Factory New)
13400000
Sticker  Vox Eminor (Holo)  Katowice 2014
13000000
Sticker  Team Dignitas (Holo)  Katowice 2014
11886000
StatTrak™ M9 Bayonet  Case Hardened (Factory New)
10999900
StatTrak™ Paracord Knife  Crimson Web (Factory New)
10000000
StatTrak™ Ursus Knife  Fade (Minimal Wear)
10000000
Sport Gloves  Slingshot (Factory New)
10000000

In post request, you have to add data which is sent by the server as payload as json response and captcha Token is also a part of payload data. Here is an example how to pull data (name, price) from api calls json response as post method.

import requests
import json
URL = "https://public-api.pricempire.com/api/search/items"

body= {"page":1,"priceMin":0,"orderBy":"price_desc","captchaToken":"03AGdBq271Msp7k_yCTzgNsheZ1yRqLWykDZL17tIK9_YAVo2uZGc3cLH0sNhuZOFsnymBSAbuzRRo2w_Cy6kEEMxaRxgkuZUlXFcDzRPWgYs-Hy-fV5SpxLjU8rACYW3KwZ8y-js1Dye8weAdMfZSPeEBgQ9YP3zdbaPrUOJAHHmjkpqTxH7vPW-Cd2PXHtZf5NlgVkxCBUKIESAyMJ6FyKdNz_WxYdIJvK4uQa6nBdHxMlmQZx6rUgus65NxZkwTaY3BO36ju68WNerv-fQBqFdIz_6jUPfav41DYFiApv9O-MbdASQqpS-ma1TG76mQ82OQdzkqqvpZtAksBGa836HzsxfaOecgbZ2YbswAHr1dXxl919DbRnZum4Wr-UUZMQ66j8Iy5UA_g4B3Ir7IxTf50KhTOrNHtqIIYuBR4Vfz6scc5c7XqATeqMoMvL-06wbBWVATSI44","priceMax":200000,"collections":[],"weaponIds":[],"wears":[],"priceProvider":"buff163"}

headers={
    'content-type': 'application/json',
    'User-Agent':'mozila/5.0/'
    }
jsonData=requests.post(URL,headers=headers,data=json.dumps(body)).json()


for item in jsonData['items']:
    name= item['name'].replace('★','').replace('|','').strip()
    price=item['price']['price']
    print(name)
    print(price)

Output:

Souvenir AWP  Dragon Lore (Minimal Wear)
100000000
Sticker  iBUYPOWER (Holo)  Katowice 2014    
49999900
Sticker  Titan (Holo)  Katowice 2014        
49999800
Souvenir AWP  Dragon Lore (Field-Tested)    
36388800
Sticker  Reason Gaming (Holo)  Katowice 2014
30000000
Souvenir AWP  Dragon Lore (Battle-Scarred)  
23618000
Sport Gloves  Pandora's Box (Factory New)
22000000
Sticker  Team LDLC.com (Holo)  Katowice 2014
16750000
StatTrak™ Ursus Knife  Crimson Web (Factory New)
15000000
StatTrak™ Talon Knife  Crimson Web (Factory New)
15000000
StatTrak™ Nomad Knife  Safari Mesh (Battle-Scarred)
15000000
Survival Knife  Crimson Web (Factory New)
14999999
StatTrak™ Stiletto Knife  Slaughter (Field-Tested)
14888800
Sport Gloves  Vice (Factory New)
13400000
Sticker  Vox Eminor (Holo)  Katowice 2014
13000000
Sticker  Team Dignitas (Holo)  Katowice 2014
11886000
StatTrak™ M9 Bayonet  Case Hardened (Factory New)
10999900
StatTrak™ Paracord Knife  Crimson Web (Factory New)
10000000
StatTrak™ Ursus Knife  Fade (Minimal Wear)
10000000
Sport Gloves  Slingshot (Factory New)
10000000
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文