Modifier and Type | Method and Description |
---|---|
Downloader |
Crawler.getDownloader()
获取网页下载器
|
Modifier and Type | Method and Description |
---|---|
Crawler |
Crawler.setDownloader(Downloader downloader)
设置网页下载器
|
static SimulatorData |
Crawler.testContent(String url,
SiteRule siteRule,
ExtractRule contentExtractRule,
Downloader downloader)
测试内容提取规则
使用自定义下载器 |
static SimulatorData |
Crawler.testDown(String url,
SiteRule siteRule,
Downloader downloader)
测试网页下载器
使用自定义下载器 |
static SimulatorData |
Crawler.testLink(String url,
SiteRule siteRule,
LinkRule linkRule,
Downloader downloader)
测试链接提取规则
使用自定义下载器 |
static SimulatorData |
Crawler.testMatcher(String url,
SiteRule siteRule,
ContentRule content,
Downloader downloader)
内容匹配测试
|
Constructor and Description |
---|
CrawlerWorker(Request request,
Downloader downloader,
CrawlerProcessor crawlerProcessor) |
Modifier and Type | Class and Description |
---|---|
class |
BaseDownloader
selenium下载器基类
所有基于selenium的下载器最好根据此基类完成 |
Modifier and Type | Class and Description |
---|---|
class |
SeleniumDownloader
基于Firefox的下载器
使用selenium-java实现 |
class |
SimpleDownloader
基于JSOUP实现的网页下载器
特性如下: 1. |
Modifier and Type | Method and Description |
---|---|
SimulatorData |
Simulator.down(String url,
SiteRule siteRule,
Downloader downloader)
测试网页下载功能
|
SimulatorData |
SimpleSimulator.down(String url,
SiteRule siteRule,
Downloader downloader) |
SimulatorData |
Simulator.extract(String url,
SiteRule siteRule,
ExtractRule contentExtractRule,
Downloader downloader)
提取测试
|
SimulatorData |
SimpleSimulator.extract(String url,
SiteRule siteRule,
ExtractRule contentExtractRule,
Downloader downloader) |
SimulatorData |
Simulator.link(String url,
SiteRule siteRule,
LinkRule linkRule,
Downloader downloader)
测试链接提取
|
SimulatorData |
SimpleSimulator.link(String url,
SiteRule siteRule,
LinkRule linkRule,
Downloader downloader) |
SimulatorData |
Simulator.match(String url,
SiteRule siteRule,
ContentRule content,
Downloader downloader)
测试内容匹配
|
SimulatorData |
SimpleSimulator.match(String url,
SiteRule siteRule,
ContentRule content,
Downloader downloader) |
Copyright © 2020 Pivotal Software, Inc.. All rights reserved.