错误信息
Exception in thread "main" java.lang.IllegalArgumentException: Cannot locate declared field class org.apache.http.impl.client.HttpClientBuilder.sslcontext
at org.apache.commons.lang3.Validate.isTrue(Validate.java:155)
at org.apache.commons.lang3.reflect.FieldUtils.readDeclaredField(FieldUtils.java:471)
at com.gargoylesoftware.htmlunit.HttpWebConnection.createConnectionManager(HttpWebConnection.java:775)
at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:159)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1313)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1230)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:338)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:407)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:392)
at accounts.crawler.util.test.main(test.java:33)
代码
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24);
//设置webClient的相关参数
webClient.getOptions().setJavaScriptEnabled(true);
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.getOptions().setThrowExceptionOnScriptError(false);
//模拟浏览器打开一个目标网址
HtmlPage rootPage = null;
try {
rootPage = webClient.getPage("http://www.yanyulin.info");
} catch (FailingHttpStatusCodeException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("为了获取js执行的数据 线程开始沉睡等待");
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}//主要是这个线程的等待 因为js加载也是需要时间的
System.out.println("线程结束沉睡");
String html = rootPage.asText();
System.out.println(html);
已解决。主要问题还是jar包不够。jar包 commons-collections-3.2.1.jar commons-io-2.4.jar commons-lang-2.6.jar cssparser-0.9.14.jar httpclient-4.3.3.jar httpcore-4.3.2.jar httpmime-4.3.3.jar nekohtml-1.9.21.jar
xalan-2.7.1.jarxercesImpl-2.11.0.jar htmlunit-2.15-OSGi.jar htmlunit-2.15.jar htmlunit-core-js-2.15.jar