Java_爬虫，如何抓取Js动态生成数据的页面？
0
悬赏园豆：20
[已关闭问题]
解决于 2014-07-29 09:46
很多网站是用js或Jquery 生成数据的，到后台获取到数据以后，用 document.write()或者("#id").html="" 的方式 写到页面中，这个时候用浏览器查看源码是看不到数据的。
HttpClient是不行的，看网上说HtmlUnit，说 可以获取后台js加载完后的完整页面，但是我按照文章上说的 写了 ，都不好使。
String url = "http://xinjinqiao.tprtc.com/admin/main/flrpro.do"; try { WebClient webClient = new WebClient(BrowserVersion.FIREFOX_10); //设置webClient的相关参数 webClient.getOptions().setJavaScriptEnabled(true); webClient.getOptions().setCssEnabled(false); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); //webClient.getOptions().setTimeout(50000); webClient.getOptions().setThrowExceptionOnScriptError(false); //模拟浏览器打开一个目标网址 HtmlPage rootPage = webClient.getPage(url); System.out.println("为了获取js执行的数据 线程开始沉睡等待"); Thread.sleep(3000);//主要是这个线程的等待 因为js加载也是需要时间的 System.out.println("线程结束沉睡"); String html = rootPage.asText(); System.out.println(html); } catch (Exception e) { }
其实这段代码不好使。
求解答，其中典型的就是这个链接的页面，怎么能在java程序中获取其中的数据？
http://xinjinqiao.tprtc.com/admin/main/flrpro.do
其他回答(1)
0
可以参考下这个文档说明，很详细
http://doc.shenjianshou.cn/developmentSkills/useAJAX.html
-1
[{"id":2081,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园32号楼房产","fdate":"2017-10-31","fmoney":"256.896900","fcode":"GR2017TJ1000233-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2080,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园31号楼房产","fdate":"2017-10-31","fmoney":"306.939500","fcode":"GR2017TJ1000231-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2079,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园27号楼房产","fdate":"2017-10-31","fmoney":"281.428600","fcode":"GR2017TJ1000232-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2078,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园26号楼房产","fdate":"2017-10-31","fmoney":"281.428600","fcode":"GR2017TJ1000327-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2077,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园19号楼房产","fdate":"2017-10-31","fmoney":"226.123900","fcode":"GR2017TJ1000326-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2076,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园18号楼房产","fdate":"2017-10-31","fmoney":"226.123900","fcode":"GR2017TJ1000228-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2075,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园5号楼房产","fdate":"2017-10-31","fmoney":"306.939500","fcode":"GR2017TJ1000230-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2074,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园4号楼房产","fdate":"2017-10-31","fmoney":"256.896900","fcode":"GR2017TJ1000229-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2073,"count":67,"fname":"天津市北辰区双口镇京福公路西侧林里庄园2号楼房产","fdate":"2017-10-31","fmoney":"290.650300","fcode":"GR2017TJ1000328-3","fproducttype":"实物资产-房产","fmediadate":null},{"id":2072,"count":67,"fname":"天津市宝坻区大白庄镇锦绣香江花园玉兰园3-2-502房产","fdate":"2017-10-31","fmoney":"59.563000","fcode":"GR2017TJ1000323-3","fproducttype":"实物资产-房产","fmediadate":null}]
