在代码中对超时设置1S 实际大于1S后才会进入超时的隐藏,所有连接都使用了using并显式关闭了,实际代码如图所示:
代码如下:
using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Net; using System.Runtime.CompilerServices; using System.Text; using HtmlAgilityPack; namespace ConsoleApplication1 { class Program { static void Main(string[] args) { HtmlWeb webClient = new HtmlWeb(); webClient.PreRequest = request => { request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate"; request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip; return true; }; var proxtUrl = "http://www.xicidaili.com/nn/"; var doc = webClient.Load(proxtUrl); var nodes = doc.DocumentNode.SelectSingleNode("/html/body/div[1]/div[2]/table").SelectNodes("tr[position()>1]"); Console.WriteLine("共计{0}条记录", nodes.Count); foreach (var node in nodes) { var items = node.SelectNodes("td"); if (items[6].InnerText.Trim() == "HTTP") { Stopwatch stopwatch = new Stopwatch(); try { stopwatch.Start(); //验证代理是否可以使用 var address = items[2].InnerText.Trim() + ":" + items[3].InnerText.Trim(); Console.WriteLine("代理地址:{0}", address); HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://1111.ip138.com/ic.asp"); webRequest.Timeout = 1000; webRequest.ReadWriteTimeout = 1000; webRequest.AllowAutoRedirect = false; webRequest.Proxy = new WebProxy(address); using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse()) { var stream = webResponse.GetResponseStream(); StreamReader reader = new StreamReader(stream); Console.WriteLine(reader.ReadToEnd()); reader.Close(); stream.Close(); webResponse.Close(); } } catch (Exception ex) { Console.WriteLine(ex.Message); } stopwatch.Stop(); Console.WriteLine("共计耗时{0}毫秒", stopwatch.Elapsed); Console.WriteLine("=========================="); } } } } }
引用HtmlAgilityPack代码即可直接运行,本问题之前在博问中问到过,但是答案中的回复没能有效的解决,之前的博问链接
主要功能就是采集免费代理网中的代理,然后筛选出有效的代理处理,思路中目前想通过设置超时时间处理,还请博友能帮忙,豆子不多,所有的都在这了。。。
https://msdn.microsoft.com/zh-cn/library/system.net.httpwebrequest.timeout.aspx
看 小心 下面的说明,自带的超时基本不准的,所以自己实现超时吧