1clickcrawl 中文文档教程
1ClickCrawler
用于 nodejs 的简单快速网络爬虫。 提供您自己的访问功能,其余的将由爬虫完成。
Installation
$ npm install 1clickcrawler
Usage
var Crawler = require('1clickcrawler');
var crawler = new Crawler(delay, seeds, visit);
crawler.start()
Parameters
延迟:请求之间等待的毫秒数(例如:200) seeds:开始抓取的 url 数组(例如:['https://example.com']) 访问:一个函数,它将使用参数 url、res、html 处理数据(例如:function(url,res,html) {})
1ClickCrawler
Simple quick webcrawler for nodejs. Provide your own visit function and the crawler will do the rest.
Installation
$ npm install 1clickcrawler
Usage
var Crawler = require('1clickcrawler');
var crawler = new Crawler(delay, seeds, visit);
crawler.start()
Parameters
delay: milliseconds to wait between requests (ex: 200) seeds: array of urls to start the crawl (ex: ['https://example.com']) visit: a function that will process the data with params url, res, html (ex: function(url,res,html) {})