Java Web Scraper项目正在返回NULL，而不是普通链接

Question

用于WebCraper的HTMLUNIT依赖性。 主要问题是我的刮板返回空而不是链接。我做了一个项目课程。

import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import java.util.List;
public class Scraper {

private static final String searchUrl = "https://sfbay.craigslist.org/search/sss?query=iphone%208&sort=rel";

public static void main(String[] args){
        WebClient client = new WebClient();
        client.getOptions().setJavaScriptEnabled(false);
        client.getOptions().setCssEnabled(false);
        client.getOptions().setUseInsecureSSL(true);

        HtmlPage page = client.getPage(searchUrl);
        List<HtmlElement> items = page.getByXPath("//li[@class='result-row']");
        for(HtmlElement htmlItem : items){

             HtmlAnchor itemAnchor = ((HtmlAnchor)htmlItem.getFirstByXPath("//a[@class='result-image gallery']")); //itemAnchor gets the anchor specified by class result-image gallery//
             Item item = new Item();
             String link = itemAnchor.getHrefAttribute(); //link is extracted and initialized in string//
             item.setUrl(link); 
             System.out.println(item.getUrl()); //why don't you work//

}

}

结果： 基本上是一条无效的行

*注：putting system.out.println（link）返回一个链接，并重新使用与打印新行相同的链接，在这种情况下，它将只是链接'https：//sfbay.craigslist 。

​任何帮助都是有用的。 编辑：以防万一，我将在此处包含依赖码

        <dependency>
            <groupId>net.sourceforge.htmlunit</groupId>
            <artifactId>htmlunit</artifactId>
            <version>2.60.0</version>
        </dependency>

Answer 1

在这里起作用，

public static void main(String[] args) throws IOException {
    String url = "https://sfbay.craigslist.org/search/sss?query=iphone%208&sort=rel";

    try (final WebClient webClient = new WebClient()) {
        HtmlPage page = webClient.getPage(url);
        // webClient.waitForBackgroundJavaScript(10_000);

        List<HtmlElement> items = page.getByXPath("//li[@class='result-row']");
        for(HtmlElement htmlItem : items){
             HtmlAnchor itemAnchor = ((HtmlAnchor)htmlItem.getFirstByXPath("a[@class='result-image gallery']"));
             if (itemAnchor != null) {
               String link = itemAnchor.getHrefAttribute();
               System.out.println("-> " + link);
             }
        }
    }
}

产生了类似

-> https://sfbay.craigslist.org/eby/pho/d/walnut-creek-original-new-defender/7470991009.html
-> https://sfbay.craigslist.org/eby/pho/d/walnut-creek-original-new-defender/7471913572.html
-> https://sfbay.craigslist.org/eby/pho/d/walnut-creek-original-new-defender/7471010388.html
....