网络刮擦mutiple Pages r，r bind Broke Broke Broke Webscraping

Question

我正在网站上贴上网站，当我尝试将数据集中的所有列引用时，我会遇到麻烦。它给了我错误。它之所以这样说，是因为数据集中的列之间的行数不同，例如价格为25个元素，描述中有24个元素。

{if(length(.) == 0) NA else .}

当Webscrape程序找不到值时，我试图将上面的代码放置在上面放置NAS，但看起来它不起作用，我将完整的代码留在下面。

urls <- sprintf("https://www.immobiliare.it/vendita-case/milano/?pag=%d", 1:7)

case <- data.frame() 

for (i in urls){
  page <- read_html(i)
  
  Scrape <- page  %>% html_nodes(xpath= "//ul[@class='nd-list in-realEstateResults']") %>% 
    purrr::map_df(~list(description= html_nodes(.x, xpath= "//a[@class='in-card__title']") %>% html_text() %>% length() %>% {if(length(.) == 0) NA else .}, #Returns NA for missing data
                        
                        price= html_nodes(.x, xpath= "//li[@class='nd-list__item in-feat__item in-feat__item--main in-realEstateListCard__features--main']") %>% html_text(trim = TRUE) %>% {if(length(.) == 0) NA else .},
                        
                        rooms = html_nodes(.x,xpath= "//li[@aria-label='locali']") %>% html_text(trim = TRUE) %>% {if(length(.) == 0) NA else .},
                        
                        area= html_nodes(.x,xpath= "//li[@aria-label='superficie']") %>% html_text(trim = TRUE) %>% {if(length(.) == 0) NA else .}))
  
  
  temp <- data.frame(Scrape)
  case <- rbind(temp, case)
  
  print(paste("Page:",i))
}

有什么建议吗？ 让我知道您是否有任何疑问

Answer 1

如果您查看我在评论中提到的问题。
您面临的问题并非每个清单节点都在您要寻找的所有信息中都有，从而产生非相等长度错误。

处理这些情况的最佳方法是在列表/向量中找到所有父节点，然后使用html_node（没有“ s”）函数。 html_node（）即使是na，每个节点也将始终返回1个结果。 html_nodes（）将不返回。
有关更多信息，请参见评论。

library(rvest)
library(dplyr)

df_apartments <- list()
for (i in 1:7) { 
   #read page
   page <- read_html(paste0("https://www.immobiliare.it/vendita-case/milano/?pag=", i))
   
   #read the parent nodes
   apartments <- page  %>% html_nodes(xpath= "//div[@class='nd-mediaObject__content in-card__content in-realEstateListCard__content']")
   
# parse information from each of the parent nodes
  price <- apartments %>% html_node(xpath= ".//li[@class='nd-list__item in-feat__item in-feat__item--main in-realEstateListCard__features--main']") %>% html_text(trim = TRUE)
  rooms <- apartments %>% html_node(xpath= ".//li[@aria-label='locali']") %>% html_text(trim = TRUE)
  area <- apartments %>% html_nodes(xpath= ".//li[@aria-label='superficie']") %>% html_text(trim = TRUE)
  description <-  apartments %>% html_node( xpath= ".//a[@class='in-card__title']") %>% html_text()
      
# put the data together into a data frame add to list                  
   df_apartments[[i]] <- data.frame(price, rooms, area, description)
}
#combine all data frames into 1
answer <- bind_rows(df_apartments)


df_apartments
         price rooms  area                                                            description
1    € 900.000     3 125m²                     Trilocale via Orti 2, Quadronno - Crocetta, Milano
2    € 275.000     2  50m²                Bilocale via Gian Francesco Pizzi 34, Ripamonti, Milano
3    € 275.000     2  55m²                 Bilocale viale dei Mille 14, Plebisciti - Susa, Milano
4    € 799.000     4 135m²                 Quadrilocale via Beato Angelico 3, Città Studi, Milano
5    € 210.000     2  65m²                    Bilocale piazza Monte Falterona 5, San Siro, Milano
6    € 240.000     2  50m²                Bilocale via dell'Assunta 5, Vigentino - Fatima, Milano
7    € 395.000     3  90m² Trilocale via Cuore Immacolato di Maria 12, Vigentino - Fatima, Milano

更新
因为我们使用的是XPATH选项，所以我们需要添加一个“”。在“ //”之前，请告诉XPath解析器以当前节点启动并注意第一个节点。