Jsoup 在指定标签之后开始解析还是从页面底部开始?

发布于 2024-12-25 07:36:32 字数 2907 浏览 2 评论 0原文

我有一个正在用 Jsoup 解析的 HTML 块,但是,并非所有内容都是相关的,并且解析不相关的部分会丢弃我的数据集。

网站上有一个可以随时更改的标题。此标题中包含链接,但我不关心链接。当 Jsoup 解析文档时,它将这些想法添加到我的链接数组中并丢弃我的值。

我感兴趣的 HTML 出现在 标签。

我希望能够告诉 Jsoup 忽略该标签上方的所有内容。这可能吗?如果没有,我可以通过在文档底部开始解析来解决这个问题,但我也不确定如何解决这个问题。

我的 Jsoup 查询如下。请忽略所有注释掉的行和调试语句,我已经尝试解决这个问题有一段时间了,并且仍然有测试代码。

       Thread getTitlesThread = new Thread() {
            public void run() {
                TitleResults titleArray =  new TitleResults();
                StringBuilder whole = new StringBuilder();

                try {
                    URL url = new URL(
                            Constants.FORUM);
                    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
                    urlConnection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2");
                    try {
                        BufferedReader in = new BufferedReader(
                            new InputStreamReader(new BufferedInputStream(urlConnection.getInputStream())));
                        String inputLine;
                        while ((inputLine = in.readLine()) != null)
                            whole.append(inputLine);
                        in.close();
                    } catch (IOException e) {}
                    finally {
                        urlConnection.disconnect();
                    }
                } catch (Exception e) {}
                Document doc = Parser.parse(whole.toString(), Constants.FORUM);
                Elements threads = doc.select("TOPICS > .topic_title");
                Elements authors = doc.select("a[hovercard-ref]");
//              for (Element author : authors) {
//                  authorArray.add(author.text());
//              }
//              cleanAuthors();
                if (threads.isEmpty()) {
                    Log.d("POC", "EMPTY BRO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!11");
                }
//              for (Element thread : threads) {
//                  titleArray =  new TitleResults();
//                  Log.d("POC", thread.toString());
//
//                  titleArray.setAuthorDate(authorArray.get(0));
//                  authorArray.remove(0);

                    //Thread title
//                  threadTitle = thread.text();
//                  titleArray.setItemName(threadTitle);
//                  
//                  //Thread link
//                  String threadStr = thread.attr("abs:href");
//                  String endTag = "/page__view__getnewpost"; //trim link
//                  threadStr = new String(threadStr.replace(endTag, ""));
//                  threadArray.add(threadStr);
//                  results.add(titleArray);
//              }
           } 
        };
        getTitlesThread.start();

I have a block of HTML that I am parsing with Jsoup, however, not all of it is relevant, and parsing the irrelevant parts throws off my data set.

On the site, there is a header that can change at any time. Within this header are links, but links that I don't care about. When Jsoup parses the document, it adds those thinks to my link array and throws off my values.

The HTML I am interested in comes after the
<!-- BEGIN TOPICS -->
tag.

I would like to be able to tell Jsoup to ignore everything above that tag. Is this possible? If not, I can work around this issue by beginning my parsing at the bottom of the document, but I'm not sure how I would go about that either.

My Jsoup query is as follows. Please ignore all the commented out lines and debugging statements, I've been trying to work this out for a while and still have the test code in.

       Thread getTitlesThread = new Thread() {
            public void run() {
                TitleResults titleArray =  new TitleResults();
                StringBuilder whole = new StringBuilder();

                try {
                    URL url = new URL(
                            Constants.FORUM);
                    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
                    urlConnection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2");
                    try {
                        BufferedReader in = new BufferedReader(
                            new InputStreamReader(new BufferedInputStream(urlConnection.getInputStream())));
                        String inputLine;
                        while ((inputLine = in.readLine()) != null)
                            whole.append(inputLine);
                        in.close();
                    } catch (IOException e) {}
                    finally {
                        urlConnection.disconnect();
                    }
                } catch (Exception e) {}
                Document doc = Parser.parse(whole.toString(), Constants.FORUM);
                Elements threads = doc.select("TOPICS > .topic_title");
                Elements authors = doc.select("a[hovercard-ref]");
//              for (Element author : authors) {
//                  authorArray.add(author.text());
//              }
//              cleanAuthors();
                if (threads.isEmpty()) {
                    Log.d("POC", "EMPTY BRO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!11");
                }
//              for (Element thread : threads) {
//                  titleArray =  new TitleResults();
//                  Log.d("POC", thread.toString());
//
//                  titleArray.setAuthorDate(authorArray.get(0));
//                  authorArray.remove(0);

                    //Thread title
//                  threadTitle = thread.text();
//                  titleArray.setItemName(threadTitle);
//                  
//                  //Thread link
//                  String threadStr = thread.attr("abs:href");
//                  String endTag = "/page__view__getnewpost"; //trim link
//                  threadStr = new String(threadStr.replace(endTag, ""));
//                  threadArray.add(threadStr);
//                  results.add(titleArray);
//              }
           } 
        };
        getTitlesThread.start();

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

や莫失莫忘 2025-01-01 07:36:32

根据您的描述,这应该可行(没有实际的 HTML 输入很难确定):

    Document document = ...;
    Elements elements = document.getAllElements();
    Element comment = null;
    int size = elements.size();
    for (int i = 0; comment == null && i < size; i++) {
        Element element = elements.get(i);
        for (Node node : element.childNodes()) {
            if (node instanceof Comment) {
                String str = ((Comment) node).getData().trim();
                if ("BEGIN TOPICS".equals(str)) {
                    comment = element;
                    break;
                }
            }
        }
    }

    // Did we find <-- BEGIN TOPICS -->?
    if (comment != null) {
        // You can now select from the siblingElements of comment
        // and only get stuff "after" that comment:
        // e.g. Elements e = comment.siblingElements().select("a");
    } else {
        // Oh snap.
    }

This ought to work, given your description (hard to be certain without the actual HTML input):

    Document document = ...;
    Elements elements = document.getAllElements();
    Element comment = null;
    int size = elements.size();
    for (int i = 0; comment == null && i < size; i++) {
        Element element = elements.get(i);
        for (Node node : element.childNodes()) {
            if (node instanceof Comment) {
                String str = ((Comment) node).getData().trim();
                if ("BEGIN TOPICS".equals(str)) {
                    comment = element;
                    break;
                }
            }
        }
    }

    // Did we find <-- BEGIN TOPICS -->?
    if (comment != null) {
        // You can now select from the siblingElements of comment
        // and only get stuff "after" that comment:
        // e.g. Elements e = comment.siblingElements().select("a");
    } else {
        // Oh snap.
    }
星光不落少年眉 2025-01-01 07:36:32

删除文档中您不想解析的部分:

Document doc = Parser.parse(whole.toString().replaceAll("<!-- end ad tag -->?.*?<!-- BEGIN TOPICS -->", ""), Constants.FORUM);

其中 是我想要忽略的内容的开头,而 < ;!-- 开始主题 --> 是结束。

Remove the part of the document that you don't want to parse with:

Document doc = Parser.parse(whole.toString().replaceAll("<!-- end ad tag -->?.*?<!-- BEGIN TOPICS -->", ""), Constants.FORUM);

Where <!-- end ad tag --> was the beginning of what I wanted to ignore and <!-- BEGIN TOPICS --> was the end.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文