使用Beautifoulsoup创建Finder功能

Question

我有一个def用于Web crapinging，但是当我将属性放在变量中时，（list）BeautifulSoup无法解决它，如果我执行执行，则它会返回无，但是如果我手工放置它，则可以正常工作。

# llibreria x fer peticions html
import requests
#importem el Soup per fer busquedes al web
from bs4 import BeautifulSoup

#funcio x buscar a la web i fer codi reutilizable n
def finder(url):
    headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"}
    page = requests.get(url[0], headers=headers)
    soup = BeautifulSoup(page.content, "html.parser")
    result = soup.find(url[1])
    price = soup.find(url[2])
    price = price.replace("€","")
    price = price.replace(".","")
    price = int(price)
    print("__________ House Finded _________")
    print(result.text)
    print(price)
    print("________________________________")
    
    
habitaclia =[ "https://www.habitaclia.com/comprar-casa_pareada-en_venta_en_llanca_el_colomer_la_bateria_la_coma-llansa-i16454000001818.htm?hab=3&ordenar=precio_mas_bajo&st=3,6,8,10,12,15&f=parking&geo=p&from=list&lo=55", "h1", "'span', {'itemprop':'price'"]
finder(habitaclia)  

Answer 1

主要问题是您的字符串“'span'，{'itemprop'：'price'“在这种情况下用作一个参数，在这种情况下为标签名称。

这是在Beautifulsoup并没有以您期望的方式解析您的字符串时引起的。字符串也缺少}，

如果您真的想这样工作，请尝试更改策略并使用更多结构化信息，例如dict：

habitaclia ={'url':'https://www.habitaclia.com/comprar-casa_pareada-en_venta_en_llanca_el_colomer_la_bateria_la_coma-llansa-i16454000001818.htm?hab=3&ordenar=precio_mas_bajo&st=3,6,8,10,12,15&f=parking&geo=p&from=list&lo=55', 
             'tag_one':'h1', 
             'tag_two': 'span',
             'filter_two': {'itemprop':'price'}
            }

并在脚本中使用它：

price = soup.find(config.get('tag_two'),config.get('filter_two')).text

示例

import requests
from bs4 import BeautifulSoup

def finder(config):
    headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"}
    page = requests.get(config.get('url'), headers=headers)
    soup = BeautifulSoup(page.content, "html.parser")
    result = soup.find(config.get('tag_one'))
    price = soup.find(config.get('tag_two'),config.get('filter_two')).text
    price = price.replace("€","")
    price = price.replace(".","")
    price = int(price)
    print("__________ House Finded _________")
    print(result.text)
    print(price)
    print("________________________________")


habitaclia ={'url':'https://www.habitaclia.com/comprar-casa_pareada-en_venta_en_llanca_el_colomer_la_bateria_la_coma-llansa-i16454000001818.htm?hab=3&ordenar=precio_mas_bajo&st=3,6,8,10,12,15&f=parking&geo=p&from=list&lo=55', 
             'tag_one':'h1', 
             'tag_two': 'span',
             'filter_two': {'itemprop':'price'}
            }

finder(habitaclia)

结果

__________ House Finded _________
Casa pareada  en venta en El Colomer - La Bateria - La Coma Llançà
248000
________________________________