在向OpenSea API提出HTTP请求时,Curl给出HTML错误1020

发布于 2025-01-31 17:46:36 字数 4809 浏览 2 评论 0原文

我正在尝试向opensea.io api提出请求。当我去网络检查员时,我可以看到访问/从页面上的大量请求。当我选择一个时,右键单击,然后选择复制作为卷曲,然后可以将其粘贴到我的终端中,并且通常将数据作为输出作为输出到终端。对于几个重新说,我收到了一条有关二进制输出的消息,我可以通过修改请求来解决。例如:

curl 'https://api.opensea.io/tokens/?limit=100' \
-X 'GET' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Cache-Control: no-cache' \
-H 'Origin: https://opensea.io' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Connection: keep-alive' \
-H 'Referer: https://opensea.io/' \
-H 'Host: api.opensea.io' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45'

以上给我二进制错误消息,我通过添加 - 在结束时进行压缩来解压缩“二进制”数据并从编码标头中删除BR选项。以下请求现在在我的航站楼现在很好。

curl 'https://api.opensea.io/tokens/?limit=100' \
-X 'GET' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Cache-Control: no-cache' \
-H 'Origin: https://opensea.io' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Connection: keep-alive' \
-H 'Referer: https://opensea.io/' \
-H 'Host: api.opensea.io' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' --compressed

这一切都很好,但这并没有解决所有请求的问题。我经历了,找到了具有我正在寻找的数据的请求,但他们给出了关于不作为网站所有者的新错误。考虑以下请求:

curl 'https://api.opensea.io/graphql/' \
-X 'POST' \
-H 'Content-Type: application/json' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Host: api.opensea.io' \
-H 'Cache-Control: no-cache' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Origin: https://opensea.io' \
-H 'Content-Length: 451' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Referer: https://opensea.io/' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Connection: keep-alive' \
-H 'Cookie: _ga_9VSBF2K4BX=GS1.1.1653330281.9.1.1653332997.0; csrftoken=BVdZtaJOMRxED1ALVr79hZfFHIcUUTeNokvuFbqkb17fPoZiEqpe5Fb26Mq4RQsg; sessionid=eyJzZXNzaW9uSWQiOiI0MzJjMWVlYi0zY2Q5LTQ4Y2QtODljZS1jZWFhNzk0NzI2ZDIifQ:1ntDPZ:iRgNCzJHvxP1nDBSR90Hjx4hcpPy8UmpZl7GG6lV2e8; ajs_anonymous_id=41ec97c3-3ebf-467b-a921-a31f94abeb2f; amp_ddd6ec=yUkvg9MB9AgtD0-EafL8wO...1g3p2k0km.1g3p52466.5c.54.ag; _fbp=fb.1.1652624043939.1609498506; _ga=GA1.2.337370304.1652623932; _gid=GA1.2.1049414718.1653330282; _uetsid=9d339a80dac511ec84300fb0b22c8619; _uetvid=ebc21490d88011ec99749d8ebc9bcd13; __cf_bm=OZmIijoynqXFgy9j69FEOB2a0As_1yLXG3751dUFAO4-1653332831-0-AX1rqerC9b2mttE3Lg4rIp33aWgqCGg2fozR3+cJTaeEEJ6xgpz1/VY5OIrHCONfYfGI26n0qHHCGtxb5YDwVBw=; cf_chl_2=; cf_chl_prog=; cf_clearance=mfMY41rDtGcV.Hkkmp5dZkZUtz10Y7fXRmobKhROBlw-1653331507-0-150; _gcl_au=1.1.13890619.1653330282; __os_session=eyJpZCI6IjQzMmMxZWViLTNjZDktNDhjZC04OWNlLWNlYWE3OTQ3MjZkMiJ9; __os_session.sig=xyK0HcEq8hEtOPpbnB0ra5A18qm3t-xGKx_2YDCmObc' \
-H 'x-signed-query: d73eda68d997705a2785aa8222d5a3c5663c392d0df699f665e44fb31e14642b' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
--data-binary '{"id":"TraitsDropdownQuery","query":"query TraitsDropdownQuery(\n  $collection: CollectionSlug!\n) {\n  collection(collection: $collection) {\n    assetCount\n    numericTraits {\n      key\n      value {\n        max\n        min\n      }\n    }\n    stringTraits {\n      key\n      counts {\n        count\n        value\n      }\n    }\n    defaultChain {\n      identifier\n    }\n    id\n  }\n}\n","variables":{"collection":"boredapeyachtclub"}}' --compressed

当网页提出请求时,网站服务器会返回一个带有各种有用数据的JSON文件。但是由于某种原因,当我提出请求时,它会给我一个HTML文件,并说:

<h1>
           <span class="error-description">Access denied</span>
           <span class="code-label">Error code <span>1020</span></span>
        </h1>
        <div class="large-font">
           <p>You do not have access to api.opensea.io.</p><p>The site owner may have set restrictions that prevent you from accessing the site. Contact the site owner for access or try loading the page again.</p>
        </div>

有人可以帮助解决这个问题吗?我需要对Curl请求进行哪些更改,以便我实际上获得所需的JSON数据?我知道该页面说我不是网站所有者,这是正确的,但是为什么它会将JSON数据提供给我的浏览器,而不是通过卷曲请求给我呢?当我通过浏览器给出的所有相同的标头和cookie时,服务器如何知道我的终端和浏览器之间的区别?我注意到在cookie中有一些CF_BM和类似的cookie可以容纳一些信息,例如Unix时间戳记。我试图使用node.js和axios即时生成当前的UNIX时间戳记,但我仍然收到相同的消息,所以我相信除了cookie差异之外,还有其他事情发生。此外,我尝试从以前的请求中找到cookie值,以查看服务器是否提供了一些您必须稍后发送的信息,但我找不到一个请求之间的任何匹配值。

任何帮助都非常感谢,既可以解决此特定问题,又要解释服务器如何识别浏览器和终端之间的差异的总体过程。

I'm trying to make a request to the OpenSea.io API. When I go to the network inspector I can see a whole slew of requests that come through to/from the page. When I select one, right click, and choose copy as curl I can then paste that into my terminal and normally the data comes through as output to the terminal. For a few reqeuests, I got a message about binary output that I was able to resolve by modifying the request. For example:

curl 'https://api.opensea.io/tokens/?limit=100' \
-X 'GET' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Cache-Control: no-cache' \
-H 'Origin: https://opensea.io' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Connection: keep-alive' \
-H 'Referer: https://opensea.io/' \
-H 'Host: api.opensea.io' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45'

The above gave me the binary error message, I resolve it by adding --compressed at the end to decompress the "binary" data and removed the br option from the encoding header. The below request works just fine in my terminal now.

curl 'https://api.opensea.io/tokens/?limit=100' \
-X 'GET' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Cache-Control: no-cache' \
-H 'Origin: https://opensea.io' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Connection: keep-alive' \
-H 'Referer: https://opensea.io/' \
-H 'Host: api.opensea.io' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' --compressed

So that's all fine and dandy, but that didn't fix my issues for all of the of the requests. I went through and found the requests that have the data that I'm looking for but they give a new error about not being the website owner. Consider the below request:

curl 'https://api.opensea.io/graphql/' \
-X 'POST' \
-H 'Content-Type: application/json' \
-H 'Pragma: no-cache' \
-H 'Accept: */*' \
-H 'Host: api.opensea.io' \
-H 'Cache-Control: no-cache' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Origin: https://opensea.io' \
-H 'Content-Length: 451' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15' \
-H 'Referer: https://opensea.io/' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Connection: keep-alive' \
-H 'Cookie: _ga_9VSBF2K4BX=GS1.1.1653330281.9.1.1653332997.0; csrftoken=BVdZtaJOMRxED1ALVr79hZfFHIcUUTeNokvuFbqkb17fPoZiEqpe5Fb26Mq4RQsg; sessionid=eyJzZXNzaW9uSWQiOiI0MzJjMWVlYi0zY2Q5LTQ4Y2QtODljZS1jZWFhNzk0NzI2ZDIifQ:1ntDPZ:iRgNCzJHvxP1nDBSR90Hjx4hcpPy8UmpZl7GG6lV2e8; ajs_anonymous_id=41ec97c3-3ebf-467b-a921-a31f94abeb2f; amp_ddd6ec=yUkvg9MB9AgtD0-EafL8wO...1g3p2k0km.1g3p52466.5c.54.ag; _fbp=fb.1.1652624043939.1609498506; _ga=GA1.2.337370304.1652623932; _gid=GA1.2.1049414718.1653330282; _uetsid=9d339a80dac511ec84300fb0b22c8619; _uetvid=ebc21490d88011ec99749d8ebc9bcd13; __cf_bm=OZmIijoynqXFgy9j69FEOB2a0As_1yLXG3751dUFAO4-1653332831-0-AX1rqerC9b2mttE3Lg4rIp33aWgqCGg2fozR3+cJTaeEEJ6xgpz1/VY5OIrHCONfYfGI26n0qHHCGtxb5YDwVBw=; cf_chl_2=; cf_chl_prog=; cf_clearance=mfMY41rDtGcV.Hkkmp5dZkZUtz10Y7fXRmobKhROBlw-1653331507-0-150; _gcl_au=1.1.13890619.1653330282; __os_session=eyJpZCI6IjQzMmMxZWViLTNjZDktNDhjZC04OWNlLWNlYWE3OTQ3MjZkMiJ9; __os_session.sig=xyK0HcEq8hEtOPpbnB0ra5A18qm3t-xGKx_2YDCmObc' \
-H 'x-signed-query: d73eda68d997705a2785aa8222d5a3c5663c392d0df699f665e44fb31e14642b' \
-H 'X-BUILD-ID: da14c5fd3811187c88141eb116061b5f6cf87f45' \
-H 'X-API-KEY: 2f6f419a083c46de9d83ce3dbe7db601' \
--data-binary '{"id":"TraitsDropdownQuery","query":"query TraitsDropdownQuery(\n  $collection: CollectionSlug!\n) {\n  collection(collection: $collection) {\n    assetCount\n    numericTraits {\n      key\n      value {\n        max\n        min\n      }\n    }\n    stringTraits {\n      key\n      counts {\n        count\n        value\n      }\n    }\n    defaultChain {\n      identifier\n    }\n    id\n  }\n}\n","variables":{"collection":"boredapeyachtclub"}}' --compressed

When the webpage makes the request, the site server returns back a JSON file with all kinds of useful data inside. But for some reason when I make the request it gives me back an HTML file and says:

<h1>
           <span class="error-description">Access denied</span>
           <span class="code-label">Error code <span>1020</span></span>
        </h1>
        <div class="large-font">
           <p>You do not have access to api.opensea.io.</p><p>The site owner may have set restrictions that prevent you from accessing the site. Contact the site owner for access or try loading the page again.</p>
        </div>

Can anybody help in resolving this? What changes do I need to make to the curl request so that I actually get the JSON data I'm looking for? I understand the page is saying that I am not the website owner and that's correct, but then why does it give the JSON data to my browser and not to me through a CURL request? How does the server know the difference between my terminal and a browser making a request when I pass through all of the same headers and cookies that the browser had given it? I noticed that in the cookies there was some cf_bm and similar cookies that hold some info like a unix time stamp. I tried to pass along the current unix time stamp generating on the fly using NODE.js and Axios but I still got the same message so I believe there's something more going on besides a cookie difference. Additionally, I tried finding the cookie values from previous requests to see if maybe the server gave it some info that you have to send back later but I couldn't find any matching values between one request to the next.

Any help is much appreciated, both in fixing this specific problem as well as explaining the overall process of how the server identifies the differences between browser and terminal.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

澜川若宁 2025-02-07 17:46:36

目标源在IP或用户代理级别上阻止您

的原因访问被拒绝1020 >并设置您的请求标题随机。

Reason Of Access Denied or 1020 is The target source is blocking you on ip or User Agent Level

Solution: Use Proxy And Set your Request Header random.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文