为什么使用Urllib和BS4有HTTP错误503？

发布于 2025-02-11 06:50:22 字数 448 浏览 1 评论 0原文

我使用BS4从网站上从技术获取浏览标准： https://standards.globals.globalspec.com/

from urllib.request import urlopen
from bs4 import BeautifulSoup
url = "https://standards.globalspec.com/"
q1 = urlopen(url)
soup = BeautifulSoup(q1, 'lxml')
print(soup)

但是但是我有一个错误：urllib.error.httperror：http错误503：服务暂时不可用，

任何人都可以看到可能导致此错误的原因吗？

原文

I use BS4 to get Browse Standards by Technology from website: https://standards.globalspec.com/

from urllib.request import urlopen
from bs4 import BeautifulSoup
url = "https://standards.globalspec.com/"
q1 = urlopen(url)
soup = BeautifulSoup(q1, 'lxml')
print(soup)

But i have an error: urllib.error.HTTPError: HTTP Error 503: Service Temporarily Unavailable

Could anyone see what could be causing this error?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

岁月静好 2025-02-18 06:50:23

@samt94已经表示该网站受到CloudFlare的保护。因此，您可以使用CloudScraper而不是请求

from bs4 import BeautifulSoup
import cloudscraper
scraper = cloudscraper.create_scraper(delay=10,   browser={'custom': 'ScraperBot/1.0',})
url = 'https://standards.globalspec.com/'
req = scraper.get(url)
print(req)
soup = BeautifulSoup(req.text,'lxml')

output：

  <Response [200]>

cloudscraper

@Samt94 already has stated that the website is under cloudflare protection. So you can use cloudscraper instead of requests

from bs4 import BeautifulSoup
import cloudscraper
scraper = cloudscraper.create_scraper(delay=10,   browser={'custom': 'ScraperBot/1.0',})
url = 'https://standards.globalspec.com/'
req = scraper.get(url)
print(req)
soup = BeautifulSoup(req.text,'lxml')

Output:

  <Response [200]>

cloudscraper

回复收藏 0 原文

呆头 2025-02-18 06:50:23

您可以使用CloudScraper访问使用CloudFlare DDOS保护的网站：

from bs4 import BeautifulSoup
import cloudscraper

url = "https://standards.globalspec.com/"


scraper = cloudscraper.create_scraper()
q1 = scraper.get(url)
soup = BeautifulSoup(q1.text, 'lxml')
print(soup)

You can use CloudScraper to access websites that use CloudFlare DDoS Protection:

from bs4 import BeautifulSoup
import cloudscraper

url = "https://standards.globalspec.com/"


scraper = cloudscraper.create_scraper()
q1 = scraper.get(url)
soup = BeautifulSoup(q1.text, 'lxml')
print(soup)

回复收藏 0 原文

~没有更多了~