首页 新闻 会员 周边 捐助

请教快速访问网址列表的方案

0
悬赏园豆:100 [已解决问题] 解决于 2010-02-23 14:21

 诚心请教各位一个问题,我在站点里面使用了缓存,只要访问一次带有指定参数的网址后,并且网页里的ajax全部加载完后,就将这些参数作为键值缓存到字典中一天时间,当本天访问带有这些参数的网址时,就直接从缓存中读取数据。但是,现在有很多条网址,需要一个比较快速可行的访问方案,现暂有以下方案:

1.直接使用批处理,但是,使用例如傲游这类多标签浏览器,一次只能打开一百个标签,但是我随便一个网址列表文件就有上千条网址记录

2.编写winform程序,使用webbrowser循环浏览,但是速度太慢了。

请教一下各位,有没有比较好的方案,先谢谢了!

hiproject的主页 hiproject | 初学一级 | 园豆:4
提问于:2010-02-21 14:41
< >
分享
最佳答案
0

呵呵,倒没有那么复杂了,直接发起HttpRequest请求就好了,这就相当于一次访问了,不用WebBrowser控件的。
PS:楼主不是要刷网站流量吧?

 

参考代码:

       public static string SendParamsByPost(Uri url, string data, ref CookieContainer cookie)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            if (cookie.Count == 0)
            {
                request.CookieContainer = new CookieContainer();
                cookie = request.CookieContainer;
            }
            else
            {
                request.CookieContainer = cookie;
            }

            byte[] bytes = Encoding.ASCII.GetBytes(data);

            request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)";
            request.Method = "POST";
            request.ContentType = "application/x-www-form-urlencoded";
            request.ContentLength = bytes.Length;
            Stream requestStream = request.GetRequestStream();
            requestStream.Write(bytes, 0, bytes.Length);

            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                Stream responseStream = response.GetResponseStream();
                cookie.Add(response.Cookies);
                using (StreamReader sr = new StreamReader(responseStream, Encoding.Default))
                {
                    string ret = sr.ReadToEnd();
                    return ret;
                }
            }
        }

        public static string SendParamsByGet(Uri url, string data, ref CookieContainer cookie)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + (data == "" ? "" : "?") + data);
            if (cookie.Count == 0)
            {
                request.CookieContainer = new CookieContainer();
                cookie = request.CookieContainer;
            }
            else
            {
                request.CookieContainer = cookie;
            }

            request.Method = "GET";
            //request.ContentType = "text/html;charset=UTF-8";

            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                Stream responseStream = response.GetResponseStream();
                cookie.Add(response.Cookies);

                using (StreamReader sr = new StreamReader(responseStream, Encoding.Default))
                {
                    string ret = sr.ReadToEnd();
                    return ret;
                }
            }
        }

收获园豆:90
查尔斯 | 老鸟四级 |园豆:3832 | 2010-02-21 15:16
3q first,我呆会试试,这样直接发起httprequest请求的速度怎么样??还有绝对不是刷流量了,刷流量可能直接去找相应的工具会快得多哈,再次感谢!
hiproject | 园豆:4 (初学一级) | 2010-02-21 15:45
其实是一样的,我给你一个代码段参考一下,之前我做的case里面的 public static string SendParamsByPost(Uri url, string data, ref CookieContainer cookie) { HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); if (cookie.Count == 0) { request.CookieContainer = new CookieContainer(); cookie = request.CookieContainer; } else { request.CookieContainer = cookie; } byte[] bytes = Encoding.ASCII.GetBytes(data); request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"; request.Method = "POST"; request.ContentType = "application/x-www-form-urlencoded"; request.ContentLength = bytes.Length; Stream requestStream = request.GetRequestStream(); requestStream.Write(bytes, 0, bytes.Length); using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { Stream responseStream = response.GetResponseStream(); cookie.Add(response.Cookies); using (StreamReader sr = new StreamReader(responseStream, Encoding.Default)) { string ret = sr.ReadToEnd(); return ret; } } } public static string SendParamsByGet(Uri url, string data, ref CookieContainer cookie) { HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url + (data == "" ? "" : "?") + data); if (cookie.Count == 0) { request.CookieContainer = new CookieContainer(); cookie = request.CookieContainer; } else { request.CookieContainer = cookie; } req
查尔斯 | 园豆:3832 (老鸟四级) | 2010-02-22 15:29
其他回答(1)
0

httprequest是不会加载页面中的脚本文件的,当然也不会执行javascript.

收获园豆:10
I,Robot | 园豆:9783 (大侠五级) | 2010-02-22 11:43
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册