不断抛出类型错误

发布于 2025-01-10 03:46:07 字数 1379 浏览 0 评论 0原文

有人可以帮我吗。每次运行此代码时,我都会得到

回溯(最近一次调用最后一次): 文件“C:\Users\rygra\Documents\Ryan Projects\totalwine-product-details-scraper\products.py”,第 24 行,位于 对于 data['product'] 中的 r: TypeError:“NoneType”对象不可迭代

在它损坏之前我让它运行了几行代码,现在它根本不会运行

from selectorlib import Extractor
import requests 
import csv

e = Extractor.from_yaml_file('totalwine_scrape.yml')

def scrape(url):    
    headers = {
        'authority': 'www.totalwine.com',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
        'referer': 'https://www.totalwine.com',
        'accept-language': 'en-US,en;q=0.9',
    }
    r = requests.get(url, headers=headers)
    return e.extract(r.text, base_url=url)

with open("urls.txt",'r') as urllist, open('data.csv','w') as outfile:
    writer = csv.DictWriter(outfile, fieldnames=["product_name","product_image","product_desc","product_company","product_country","product_type","product_abv","product_taste"],quoting=csv.QUOTE_ALL)
    writer.writeheader()
    for url in urllist.read().splitlines():
        data = scrape(url) 
        for r in data:
            writer.writerow(r)
           
            

Can someone help me out. Everytime I run this code I get

Traceback (most recent call last):
File "C:\Users\rygra\Documents\Ryan Projects\totalwine-product-details-scraper\products.py", line 24, in
for r in data['product']:
TypeError: 'NoneType' object is not iterable

I had it running once for a few lines of code before it broke, now it wont run at all

from selectorlib import Extractor
import requests 
import csv

e = Extractor.from_yaml_file('totalwine_scrape.yml')

def scrape(url):    
    headers = {
        'authority': 'www.totalwine.com',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36',
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
        'referer': 'https://www.totalwine.com',
        'accept-language': 'en-US,en;q=0.9',
    }
    r = requests.get(url, headers=headers)
    return e.extract(r.text, base_url=url)

with open("urls.txt",'r') as urllist, open('data.csv','w') as outfile:
    writer = csv.DictWriter(outfile, fieldnames=["product_name","product_image","product_desc","product_company","product_country","product_type","product_abv","product_taste"],quoting=csv.QUOTE_ALL)
    writer.writeheader()
    for url in urllist.read().splitlines():
        data = scrape(url) 
        for r in data:
            writer.writerow(r)
           
            

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文