首页新闻找找看学习计划

关于采集网站

0
悬赏园豆:50 [已解决问题] 解决于 2012-02-15 12:52

我想采集图片中的类似方框的类容 请问怎么做?

问题补充:

我是新手,能说具体点吗  那正则怎么写

miloss的主页 miloss | 菜鸟二级 | 园豆:254
提问于:2012-02-14 16:47
< >
分享
最佳答案
0

用正则表达式匹配

收获园豆:50
LCM | 大侠五级 |园豆:6876 | 2012-02-14 16:55

你把采集到的html发上来,然后指出你要求采集里面的哪些内容,你希望采集的内容有什么约束,比如上面的apparel,是不是必须在class名称为lists的div下的前面有dl和dt的a标签内容。这样才能写的。

LCM | 园豆:6876 (大侠五级) | 2012-02-14 17:03

@LCM:HTML就这多啊,采集每个</a>标签前面的单词啊 必须在class名称为lists的div下的前面有dl和dt的a标签内容

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:21

@LCM: 我还要将他们一次插入数据库 一数组的方式采集出来

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:22

@miloss: 我的意思是你把html用字符串的形势放上面,否则别人给你写的话连测试都不好测试,还得照着图上写一次那么长的字符串,累不累啊。

LCM | 园豆:6876 (大侠五级) | 2012-02-14 17:23

@LCM: 

<div class="lists">
  <dl><dt><a href="/categories/apparel/">Apparel</a></dt><dd><a href="/wholesale-apparel/">Apparel</a></dd><dd><a href="/wholesale-blazers+sport+coats/">Blazers & Sport Coats</a></dd><dd><a href="/wholesale-costume+cosplay/">Costume & Cosplay</a></dd><dd><a href="/wholesale-hoodies+sweatshirts/">Hoodies Sweatshirts </a></dd><dd><a href="/wholesale-jeans/">Jeans</a></dd><dd><a href="/wholesale-men+s+clothing/">Men's Clothing</a></dd><dd><a href="/wholesale-men+s+sweaters/">Men's Sweaters</a></dd><dd><a href="/wholesale-other+apparel/">Other Apparel</a></dd><dd><a href="/wholesale-other+men+s+clothing/">other men's clothing</a></dd><dd><a href="/wholesale-outerwear/">Outerwear</a></dd><dd><a href="/wholesale-pants/">Pants</a></dd><dd><a href="/wholesale-shirts/">Shirts</a></dd><dd><a href="/wholesale-shorts/">Shorts</a></dd><dd><a href="/wholesale-suits/">Suits</a></dd><dd><a href="/wholesale-swimmingwear/">Swimmingwear</a></dd><dd><a href="/wholesale-underwear/">Underwear</a></dd><dd><a href="/wholesale-vests/">Vests</a></dd><dd><a href="/wholesale-women+s+blazers+jackets/">Women's Blazers Jackets</a></dd><dd><a href="/wholesale-women+s+clothing/">Women's Clothing</a></dd><dd><a href="/wholesale-women+s+dresses/">Women's Dresses</a></dd><dd><a href="/wholesale-women+s+hosiery/">Women's Hosiery</a></dd><dd><a href="/wholesale-women+s+jeans/">Women's Jeans</a></dd><dd><a href="/wholesale-women+s+juniors/">Women's Juniors</a></dd><dd><a href="/wholesale-women+s+overcoat/">Women's Overcoat</a></dd><dd><a href="/wholesale-women+s+petites/">Women's Petites</a></dd><dd><a href="/wholesale-women+s+plus+sizes/">Women's Plus Sizes</a></dd><dd><a href="/wholesale-women+s+shirts+tops/">Women's Shirts Tops</a></dd><dd><a href="/wholesale-women+s+shorts/">Women's Shorts </a></dd><dd><a href="/wholesale-women+s+skirts/">Women's Skirts</a></dd><dd><a href="/wholesale-women+s+suits/">Women's Suits</a></dd><dd><a href="/wholesale-women+s+sweaters/">Women's Sweaters</a></dd><dd><a href="/wholesale-women+s+swimmingwear/">Women's Swimmingwear</a></dd><dd><a href="/wholesale-women+s+t+shirts+tank+tops/">Women's T-Shirts Tank Tops </a></dd><dd><a href="/wholesale-women+s+trousers/">Women's Trousers</a></dd><dd><a href="/wholesale-women+s+vests/">Women's Vests</a></dd></dl><dl><dt><a href="/categories/arts+and+crafts/">Arts and Crafts</a></dt><dd><a href="/wholesale-antique+imitation+crafts/">Antique Imitation Crafts</a></dd><dd><a href="/wholesale-arts+and+crafts/">Arts and Crafts</a></dd><dd><a href="/wholesale-bamboo+and+wooden+crafts/">Bamboo and Wooden Crafts</a></dd><dd><a href="/wholesale-basketry/">Basketry</a></dd><dd><a href="/wholesale-bead+art/">Bead Art</a></dd><dd><a href="/wholesale-candle+and+soap+making/">Candle and Soap Making</a></dd><dd><a href="/wholesale-ceramics+pottery/">Ceramics & Pottery </a></dd><dd><a href="/wholesale-embroidery+and+knitting/">Embroidery and knitting</a></dd><dd><a href="/wholesale-folk+and+native+crafts/">Folk and Native Crafts</a></dd><dd><a href="/wholesale-general+art+and+craft/">General Art and Craft</a></dd><dd><a href="/wholesale-jade+crafts/">Jade Crafts</a></dd><dd><a href="/wholesale-metal+and+tin+pieces/">Metal and Tin Pieces</a></dd><dd><a href="/wholesale-mosaic+and+glass+art/">Mosaic and Glass Art</a></dd><dd><a href="/wholesale-other+arts+and+crafts/">Other Arts and Crafts</a></dd><dd><a href="/wholesale-paintings+and+calligraphies/">Paintings and Calligraphies</a></dd><dd><a href="/wholesale-paper+crafts/">Paper Crafts</a></dd><dd><a href="/wholesale-plastic+and+resinic+crafts/">Plastic and Resinic Crafts</a></dd><dd><a href="/wholesale-religious+crafts/">Religious Crafts</a></dd><dd><a href="/wholesale-sculpture+and+carvings/">Sculpture and Carvings</a></dd><dd><a href="/wholesale-shell+crafts/">Shell crafts</a></dd></dl><dl><dt><a href="/categories/bags+shoes+accessories/">Bags Shoes & Accessories</a></dt><dd><a href="/wholesale-baby+shoes/">Baby Shoes</a></dd><dd><a href="/wholesale-backpacks+and+book+bags/">Backpacks and Book Bags</a></dd><dd><a href="/wholesale-bags+handbags/">Bags & Handbags</a></dd><dd><a href="/wholesale-bags+shoes+accessories/">Bags Shoes & Accessories</a></dd><dd><a href="/wholesale-brief+case/">Brief Case</a></dd><dd><a href="/wholesale-children+s+shoes/">Children's Shoes</a></dd><dd><a href="/wholesale-clutch+bags/">Clutch Bags</a></dd><dd><a href="/wholesale-cosmetic+bags/">Cosmetic Bags</a></dd><dd><a href="/wholesale-diaper+bags/">Diaper Bags</a></dd><dd><a href="/wholesale-evening+bags/">Evening Bags</a></dd><dd><a href="/wholesale-heels+pumps/">Heels Pumps</a></dd><dd><a href="/wholesale-hobos+bags/">Hobos Bags</a></dd><dd><a href="/wholesale-laptop+bag/">Laptop Bag</a></dd><dd><a href="/wholesale-loafers/">Loafers</a></dd><dd><a href="/wholesale-luggage+brief+case/">Luggage & Brief case</a></dd><dd><a href="/wholesale-luggage+bags+travel+bags/">Luggage Bags Travel Bags</a></dd><dd><a href="/wholesale-men+s+accessories/">Men's Accessories</a></dd><dd><a href="/wholesale-men+s+athletic/">Men's Athletic</a></dd><dd><a href="/wholesale-men+s+athletic+inspired/">Men's Athletic-Inspired</a></dd><dd><a href="/wholesale-men+s+bags/">Men's Bags</a></dd><dd><a href="/wholesale-men+s+belts+buckles/">Men's Belts & Buckles</a></dd><dd><a href="/wholesale-men+s+boots/">Men's Boots</a></dd><dd><a href="/wholesale-men+s+casual+comfort/">Men's Casual Comfort</a></dd><dd><a href="/wholesale-men+s+dress+shoes/">Men's Dress Shoes</a></dd><dd><a href="/wholesale-men+s+hats+caps/">Men's Hats & Caps</a></dd><dd><a href="/wholesale-men+s+sandals+clogs/">Men's Sandals & Clogs</a></dd><dd><a href="/wholesale-men+s+shoes/">Men's Shoes</a></dd><dd><a href="/wholesale-men+s+slippers/">Men's Slippers</a></dd><dd><a href="/wholesale-messenger+bag+sling/">Messenger Bag Sling</a></dd><dd><a href="/wholesale-moccasins/">Moccasins</a></dd><dd><a href="/wholesale-other+handbags/">Other Handbags</a></dd><dd><a href="/wholesale-other+men+s+accessories/">Other Men's Accessories</a></dd><dd><a href="/wholesale-other+men+s+shoes/">Other Men's Shoes</a></dd><dd><a href="/wholesale-other+women+s+accessories/">Other Women's Accessories</a></dd><dd><a href="/wholesale-other+women+s+shoes/">Other Women's shoes</a></dd><dd><a href="/wholesale-oxfords/">Oxfords</a></dd><dd><a href="/wholesale-platforms+wedges/">Platforms Wedges</a></dd><dd><a href="/wholesale-scarves/">Scarves</a></dd><dd><a href="/wholesale-shopping+bags/">Shopping Bags</a></dd><dd><a href="/wholesale-shoulders/">Shoulders</a></dd><dd><a href="/wholesale-ties/">Ties</a></dd><dd><a href="/wholesale-tote+bags/">Tote Bags</a></dd><dd><a href="/wholesale-wallets/">Wallets</a></dd><dd><a href="/wholesale-women+s+accessories/">Women's Accessories</a></dd><dd><a href="/wholesale-women+s+athletic/">Women's Athletic</a></dd><dd><a href="/wholesale-women+s+athletic+inspired/">Women's Athletic-Inspired</a></dd><dd><a href="/wholesale-women+s+belts+buckles/">Women's Belts & Buckles</a></dd><dd><a href="/wholesale-women+s+boots/">Women's Boots</a></dd><dd><a href="/wholesale-women+s+flats/">Women's Flats</a></dd><dd><a href="/wholesale-women+s+hats+caps/">Women's Hats & Caps</a></dd><dd><a href="/wholesale-women+s+sandals/">Women's Sandals</a></dd><dd><a href="/wholesale-women+s+shoes/">Women's Shoes</a></dd><dd><a href="/wholesale-women+s+slippers/">Women's Slippers</a></dd></dl><dl><dt><a href="/categories/car+vehicles+and+accessories/">Car,Vehicles and Accessories</a></dt><dd><a href="/wholesale-boat/">boat</a></dd><dd><a href="/wholesale-car+audio+and+video/">Car Audio and Video</a></dd><dd><a href="/wholesale-car+badge/">Car Badge</a></dd><dd><a href="/wholesale-car+gps/">Car GPS</a></dd><dd><a href="/wholesale-car+maintenance/">Car Maintenance</a></dd><dd><a href="/wholesale-car+vehicles+and+accessories/">Car,Vehicles and Accessories</a></dd><dd><a href="/wholesale-lighting+lamps+and+accessories/">Lighting Lamps and Accessories</a></dd><dd><a href="/wholesale-other+car+vehicles+and+accessories/">Other Car,Vehicles and Accessories</a></dd><dd><a href="/wholesale-parking+sensor/">Parking Sensor</a></dd><dd><a href="/wholesale-protective+gear/">Protective Gear</a></dd></dl>
  </div>
miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:25

@LCM: 太长了   不好意思哈

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:26

@miloss: 没事。等下我再看了。下班了。半个小时左右到家。

LCM | 园豆:6876 (大侠五级) | 2012-02-14 17:30

@LCM: 我也快下班啊  我等你哈

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:33

@LCM: 加我Q吧  1054509745 老鸟得帮助菜鸟哈

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 17:36

你帮我匹配第一<dl></dl>标签里面的那些吧 每个dl标签为一类 要分类插入  所以匹配条件是class为lists的div dl标签开头是<dt><a href="/categories/apparel/">Apparel</a></dt>的

miloss | 园豆:254 (菜鸟二级) | 2012-02-14 18:04

@miloss: 你试一下这个,可以匹配全部的。(?<=<div([\w\W]*?)>\s*<dl>([\w\W]*?))([\w\W]*?)<a[^>]*>(?<c>[^<]*)</a>([\w\W]*?)

LCM | 园豆:6876 (大侠五级) | 2012-02-14 18:39

@miloss: 我匹配用的C#代码是这样的,str是正则表达式,str2是html内容。

 string str = textBox1.Text.Trim();
string str2 = textBox2.Text.Trim();
Regex regex = new Regex(str, RegexOptions.None);
MatchCollection matchCollection = regex.Matches(str2);

listBox1.Items.Clear();
for (int i = 0; i < matchCollection.Count; i++)
{
Match match = matchCollection[i];
listBox1.Items.Add(match.Groups["c"].Value);
}



LCM | 园豆:6876 (大侠五级) | 2012-02-14 18:43

@LCM: 这个里面还匹配出a标签的href

(?<=<div([\w\W]*?)>\s*<dl>([\w\W]*?))([\w\W]*?)<a([^>]*?)href=\"(?<href>[\w\W]*?)\"([^>]*?)>(?<c>[^<]*)</a>([\w\W]*?)



可以这样获取到href和内容

match.Groups["c"].Value+"||"+match.Groups["href"].Value


LCM | 园豆:6876 (大侠五级) | 2012-02-14 19:02

@LCM: 这匹配出来的事空的啊  我要分别匹配出没个dl里面的     

miloss | 园豆:254 (菜鸟二级) | 2012-02-15 09:46

@miloss: 匹配出来不会是空的,你那要是看不到效果的话,等下我发给你源码。匹配出每个dl里面的,我可以改一下,等下发给你。我上午要去开个会。中午帮你弄下。你把邮箱告诉我,我等下发到你邮箱里。

LCM | 园豆:6876 (大侠五级) | 2012-02-15 09:48

@LCM: chentaolianjie@sina.com   可以加我Q嘛1054509745 我不会经常烦你的

miloss | 园豆:254 (菜鸟二级) | 2012-02-15 10:48

@miloss: 邮件已发送,里面有注释的。公司登录不了QQ的,并且也很少上QQ。

LCM | 园豆:6876 (大侠五级) | 2012-02-15 11:11

@LCM: 我看见你这没有去标签的  标签是什么去的啊

miloss | 园豆:254 (菜鸟二级) | 2012-02-15 16:35

@miloss: 我没去标签,我只匹配出我想要的内容。

LCM | 园豆:6876 (大侠五级) | 2012-02-15 16:39

@LCM:   for (int i = 0; i < matchCollection.Count; i++)
            {
                Match match = matchCollection[i];
                string name = match.Groups["ass"].Value;
                //listBox1.Items.Add(match.Groups["c"].Value);
            } 

你看我这样获取的单个  单词  怎么就是空的

miloss | 园豆:254 (菜鸟二级) | 2012-02-15 17:01

@miloss:      string url = "http://www.tootoomart.com/allcategories/";
            string  charset = "utf-8";
            string html =GetHtmlSource(url, charset);  //获取页面html
            string pattern_Apparel=@"(?<=<div([\w\W]*?)>\s*<dl>([\w\W]*?))([\w\W]*?)<a[^>]*>(?<c>[^<]*)</a>([\w\W]*?)";
            Regex regex = new Regex(pattern_Apparel, RegexOptions.None);
            MatchCollection matchCollection = regex.Matches(html);
            for (int i = 0; i < matchCollection.Count; i++)
            {
                Match match = matchCollection[i];
                string name = match.Groups["ass"].Value;
                //listBox1.Items.Add(match.Groups["c"].Value);
            } 

   我不要连接  只要那  单词

miloss | 园豆:254 (菜鸟二级) | 2012-02-15 17:05

@miloss: 你的这里面没有ass的组啊。网上查一下正则表达式怎么分组的。

LCM | 园豆:6876 (大侠五级) | 2012-02-15 17:08

@LCM: 我用变量 i  的时候name也为空啊  有时有值  我想把这name 直接先存进数据库

miloss | 园豆:254 (菜鸟二级) | 2012-02-16 09:28

@miloss: 你分组的话你的正则表达式里面要有   (?<组名>正则表达式)  这样的,你才能用match.Groups["ass"].Value读到组内的东西的。ass就是上面定义的组名。否则哪会有内容呢。

LCM | 园豆:6876 (大侠五级) | 2012-02-16 09:46
其他回答(1)
0

写爬虫的童鞋可以试试神箭手云爬虫,自带JS渲染、代理ip、验证码识别等功能,还可以发布和导出爬取的数据,生成图表等,都在云端进行,不需要安装开发环境。

sosozzzx | 园豆:302 (菜鸟二级) | 2016-09-18 17:03
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册