首页 新闻 赞助 找找看

HttpWebRequest使用代理设置超时大余实际时间的问题

0
悬赏园豆:50 [已关闭问题] 关闭于 2015-11-23 10:17

在代码中对超时设置1S 实际大于1S后才会进入超时的隐藏,所有连接都使用了using并显式关闭了,实际代码如图所示:

代码如下:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.Runtime.CompilerServices;
using System.Text;
using HtmlAgilityPack;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {


            HtmlWeb webClient = new HtmlWeb();

            webClient.PreRequest = request =>
            {
                request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
                request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
                return true;
            };


            var proxtUrl = "http://www.xicidaili.com/nn/";

            var doc = webClient.Load(proxtUrl);

            var nodes =
              doc.DocumentNode.SelectSingleNode("/html/body/div[1]/div[2]/table").SelectNodes("tr[position()>1]");
            Console.WriteLine("共计{0}条记录", nodes.Count);

            foreach (var node in nodes)
            {
                var items = node.SelectNodes("td");

                if (items[6].InnerText.Trim() == "HTTP")
                {
                    Stopwatch stopwatch = new Stopwatch();
                    try
                    {

                        stopwatch.Start();
                        //验证代理是否可以使用
                        var address = items[2].InnerText.Trim() + ":" + items[3].InnerText.Trim();
                        Console.WriteLine("代理地址:{0}", address);
                        HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://1111.ip138.com/ic.asp");


                        webRequest.Timeout = 1000;
                        webRequest.ReadWriteTimeout = 1000;
                        webRequest.AllowAutoRedirect = false;
                        webRequest.Proxy = new WebProxy(address);

                        
                        using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
                        {
                            var stream = webResponse.GetResponseStream();

                            StreamReader reader = new StreamReader(stream);
                            Console.WriteLine(reader.ReadToEnd());

                            reader.Close();
                            stream.Close();
                            webResponse.Close();

                        }

                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine(ex.Message);
                    }

                    stopwatch.Stop();
                    Console.WriteLine("共计耗时{0}毫秒", stopwatch.Elapsed);
                    Console.WriteLine("==========================");
                }
            }
        }
    }
}
View Code

引用HtmlAgilityPack代码即可直接运行,本问题之前在博问中问到过,但是答案中的回复没能有效的解决,之前的博问链接

主要功能就是采集免费代理网中的代理,然后筛选出有效的代理处理,思路中目前想通过设置超时时间处理,还请博友能帮忙,豆子不多,所有的都在这了。。。

YangChengHu的主页 YangChengHu | 初学一级 | 园豆:11
提问于:2015-11-19 18:16
< >
分享
所有回答(1)
0

https://msdn.microsoft.com/zh-cn/library/system.net.httpwebrequest.timeout.aspx

看 小心 下面的说明,自带的超时基本不准的,所以自己实现超时吧

xmj112288 | 园豆:126 (初学一级) | 2015-11-19 18:36
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册