爬虫抓取c5game饰品交易数据

本来想今天接着复习计算机网络的，结果晚饭后看到有人写抓取steam的数据，然后、、、手就痒痒起来。最近正好在igxe和c5game上倒卖饰品，平时一个个点开，真的巨麻烦

简单的写了下csgo的，只爬取了c5game的饰品的名称，最低价，周销量以及总销量，顺便把steam上该饰品的市场url也抓了出来

下面便是代码：

# -*- coding: utf-8 -*-
"""
Created on Mon Dec 04 22:38:17 2017

@author: dala_da
"""

import urllib2
import re

p=re.compile(r'''<p class="name">
<a href="(.+?)"><span class=" text-unique ">(.+?)</span></a>
</p>

<p class="info">
<span class="pull-left">
.+<span class="price">￥(.+?)</span> .+                                </span>
<span class="num">
(.+)                                </span>''')

for i in range(10):
target=('https://www.c5game.com/csgo/default/result.html?locale=zh&page=%d') %i
url1=urllib2.urlopen(target)
result_csgo_menu=url1.read()
match_csgo_menu=p.findall(result_csgo_menu)

for row in match_csgo_menu:
address=row[0]
name=row[1]
c5game_price=row[2]
count=row[3]
print name, c5game_price,count

运行结果：



2017-12-06更

好像正则表达式写的很low，重新改了下，顺便把c5game上给出的steam预测价也抓了出来，比对c5game售价和steam预测价，差价狠狠赚一波


import urllib2
import re

p=re.compile(r'''<p class="name">\s+<a href="(.+?)"><span class=" text-unique ">(.+?)</span></a>\s+</p>\s+<p class="info">\s+<span class="pull-left">\s+.+<span class="price">￥(.+?)</span> .+\s+</span>\s+<span class="num">\s+(.+)\s+</span>''')

q=re.compile(r'''<div class="hero">\s+<span>Reference: .+\( about ￥ (.+?) \)</span>''')

for i in range(10):
target=('https://www.c5game.com/csgo/default/result.html?locale=zh&page=%d') %i
url1=urllib2.urlopen(target)
result_csgo_menu=url1.read()
match_csgo_menu=p.findall(result_csgo_menu)

for row in match_csgo_menu:
address=row[0]
name=row[1]
c5game_price=float(row[2])
count=row[3]

c5game_address='https://www.c5game.com/'+address
url_c5game_thing=urllib2.urlopen(c5game_address)
result_c5game_thing=url_c5game_thing.read()
match_thing=q.findall(result_c5game_thing)

predict_price=float(match_thing[0])

print '%s\t%s' %(name,count)
print 'c5最低价：%f\t\tsteam预测价：%f' %(c5game_price,predict_price)



不过好像c5给出的预测价不太准，我尝试抓了一下steam市场的数据，发现货币种类不同，我只能够抓取我大天朝玩家出售的饰品，而天朝玩家饰品占的总数不算太多，差不多1/5，好像没什么参考性，等以后研究出怎么统一货币再更新一下