Package | Description |
---|---|
com.yishuifengxiao.common.crawler |
Modifier and Type | Method and Description |
---|---|
Crawler |
Crawler.addExtra(Map<String,Object> map)
设置风铃虫携带的额外信息
此设置不会清空原始的额外信息,而是将新的数据追加到原始的数据上 |
Crawler |
CrawlerBuilder.creatCrawler()
创建一个风铃虫简单实例
|
static Crawler |
Crawler.create(CrawlerRule crawlerRule)
创建一个默认的风铃虫实例
|
Crawler |
Crawler.setContentExtract(ContentExtract contentExtract)
设置内容解析器
|
Crawler |
Crawler.setCrawlerListener(CrawlerListener crawlerListener)
设置事件监听器
|
Crawler |
Crawler.setDownloader(Downloader downloader)
设置网页下载器
|
Crawler |
Crawler.setDuplicateRemover(DuplicateRemover duplicateRemover)
设置请求去重器
|
Crawler |
Crawler.setExtra(Map<String,Object> map)
设置风铃虫携带的额外信息
此设置会清空原始的额外信息 |
Crawler |
Crawler.setExtra(String key,
Object value)
设置风铃虫额外信息
|
Crawler |
Crawler.setLinkExtract(LinkExtract linkExtract)
设置链接解析器
|
Crawler |
Crawler.setName(String name)
设置风铃虫实例的名字
|
Crawler |
Crawler.setPipeline(Pipeline pipeline)
设置信息输出器
|
Crawler |
Crawler.setRequestCache(RequestCache requestCache)
设置资源缓存器
|
Crawler |
Crawler.setScheduler(Scheduler scheduler)
设置资源调度器
|
Crawler |
Crawler.setStatuObserver(StatuObserver statuObserver)
设置状态监听器
|
Crawler |
Crawler.setThreadPool(ThreadPoolExecutor threadPool) |
Constructor and Description |
---|
CrawlerProcessor(Crawler crawler,
ThreadPoolExecutor threadPool) |
Copyright © 2020 Pivotal Software, Inc.. All rights reserved.