使用 Webkit 包构建网络爬虫
我正在尝试构建一个网络爬虫。
我需要两件事：
- 将 HTML 转换为 DOM 对象。
- 按需执行现有的 JavaScript。
Also, I need an option to execute on demand additional JavaScripts (on events like:
onMouseOver, onMouseClick etc.) First of all, I couldn't find a good documentation source.I searched through Webkit Main Page but couldn't find much information for users of the package, and no usefull code examples. Also, in some forums I've seen instructions not to use the Webkit interface for crawlers, but directly the DOM and Javascript inner packages.I'm searching for Documentation and Code Examples.
Also, any recommendations on proper usage.
Work environment:
- 操作系统：Windows
- 语言：C++
I'm trying to build a web crawler.
I need 2 things:
- Convert the HTML into a DOM object.
- Execute existing JavaScripts on demand.
The result I expect is a DOM Object, where the JavaScript that executes on-load is already executed.
Also, I need an option to execute on demand additional JavaScripts (on events like: onMouseOver, onMouseClick etc.)
First of all, I couldn't find a good documentation source.
I searched through Webkit Main Page but couldn't find much information for users of the package, and no usefull code examples.
Also, in some forums I've seen instructions not to use the Webkit interface for crawlers, but directly the DOM and Javascript inner packages.
I'm searching for Documentation and Code Examples.
Also, any recommendations on proper usage.
Work environment:
- OS: Windows
- Lang: C++
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
查看与 WebKit 主干一起打包的一些测试工具。 大多数端口（据我所知）都包含 DumpRenderTree，它实例化 WebKitView，然后在处理指定文件后吐出渲染树。 从理论上讲，它是最简单的 WebKit 示例之一。
Check out some of the testing tools packaged alongside the WebKit trunk. Most ports (as far as I know) include DumpRenderTree which instantiates an WebKitView and then spits out a render tree after processing a specified file. It's theoretically one of the simplest examples of WebKit possible.