Openstack源代码分析之Eventlet

Keystone在创建Web服务器对外提供服务时，使用了第三方Eventlet库。

一.基本概念

官方网站对eventlet的描述是：

Eventlet is built around the concept of green threads (i.e. coroutines, we use the terms interchangeably) that are launched to do network-related work. Green threads differ from normal threads in two main ways:

Green threads are so cheap they are nearly free. You do not have to conserve green threads like you would normal threads. In general, there will be at least one green thread per network connection.
Green threads cooperatively yield to each other instead of preemptively being scheduled. The major advantage from this behavior is that shared data structures don’t need locks, because only if a yield is explicitly called can
another green thread have access to the data structure. It is also possible to inspect primitives such as queues to see if they have any pending data.
大概意思是Eventlet是以绿色线程（协同线程）的概念建立起来的网络库，绿色线程和普通线程的区别是：1.绿色线程的开销小 2.绿色线程共享数据，无需锁，同一时刻只有一个线程能访问数据，通过类似队列的去查找等待的数据。
基础API说明：

1.greenthread spawn

eventlet.spawn(func,*args,**kw) 开一个greenthread跑func

eventlet.spawn_n(func,*args,**kw)开一个greenthread跑func,但不知道greenthread何时终止，运行速度快

eventlet.spawn_after(seconds,func,*args,**kw)开一个greenthread跑func

2.greenthread control

eventlet.sleep(seconds=0) 睡眠当前greenthread

classeventlet.GreenPool 线程池管理并发线程

classeventlet.GreenPile 能批量处理大量工作

classeventlet.Queue 线程间通信

classeventlet.Timeout 线程超时处理，raise exceptions after timeout seconds.

3.patching functions

eventlet.import_patched(modulename,*additional_modules,**kw_additional_modules) 向线程导入标准模块的绿色版本

eventlet.monkey_patch(all=True,os=False,select=False,socket=False,thread=False,time=False) 全局向线程导入系统模块

4.network convenience functions

eventlet.connect(addr, family=2, bind=None) 客户端sockets，连接对象

eventlet.listen(addr,family=2, backlog=50) 服务端sockes,监听对象

eventlet.wrap_ssl(sock,*a, **kw) 配置ssl，通过PyOpenssl

eventlet.serve(sock,handle, concurrency=1000) 建立服务器，scck为listen对象，handle为处理函数，最后一个为最大并发支持数

class eventlet.StopServe 停止服务器

二.Eventlet设计模式

有三种模式client,server和dispatch模式

client模式(Wrab crawler网页爬虫)，测试urls有错误

urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
"https://wiki.secondlife.com/w/images/secondlife.jpg",
"http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]

import eventlet
from eventlet.green import urllib2

def fetch(url):
return urllib2.urlopen(url).read()

pool = eventlet.GreenPool()
for body in pool.imap(fetch, urls):
print "got body", len(body)

server模式，建立一个简单的server,接受一个字符，返回一个字符

import eventlet

def handle(client):
while True:
c = client.recv(1)
if not c: break
client.sendall(c)

server = eventlet.listen(('0.0.0.0', 6000))
pool = eventlet.GreenPool(10000)
while True:
new_sock, address = server.accept()
pool.spawn_n(handle, new_sock)

dispatch模式，处理请求多个RSS feeds，处理完，返回多个RSS feeds的数据titile

import eventlet
feedparser = eventlet.import_patched('feedparser')

pool = eventlet.GreenPool()

def fetch_title(url):
d = feedparser.parse(url)
return d.feed.get('title', '')

def app(environ, start_response):
pile = eventlet.GreenPile(pool)
for url in environ['wsgi.input'].readlines():
pile.spawn(fetch_title, url)
titles = '\n'.join(pile)
start_response('200 OK', [('Content-type', 'text/plain')])
return [titles]

三.Eventlet测试
客户端-服务器程序，发送数据，并返回发送的数据

客户端代码：

import eventlet
c=eventlet.connect(('127.0.0.1', 6000))
while True:
data=raw_input('Enter data:')
c.sendall(data)
rc=c.recv(1024)
print rc

服务端代码：

import eventlet
c=eventlet.connect(('127.0.0.1', 6000))
while True:
data=raw_input('Enter data:')
c.sendall(data)
rc=c.recv(1024)
print rc

import eventlet
def handle(client):
while True:
c = client.recv(1024)
print c
client.sendall(c)
server = eventlet.listen(('0.0.0.0', 6000))
pool = eventlet.GreenPool(10000)
while True:
new_sock, address = server.accept()
pool.spawn_n(handle, new_sock)

WSGI服务器实现

import eventlet
from eventlet import wsgi

def hello_world(env, start_response):
if env['PATH_INFO'] != '/':
start_response('404 Not Found', [('Content-Type', 'text/plain')])
return ['Not Found\r\n'
eventlet.import_patched('httplib2')
eventlet.monkey_patch(socket=True, select=True)
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello, World!\r\n']

wsgi.server(eventlet.listen(('', 8090)), hello_world)

 四.标准库在eventlet下使用
通过Import Green导入单个，或通过Monkeypatching the Standard Library导入所有的库

eventlet.import_patched('httplib2')
eventlet.monkey_patch(socket=True, select=True)

五.eventet相关模块

backdoor – Python interactive interpreter
within a running process
corolocal – Coroutine local storage
debug – Debugging tools for Eventlet
db_pool – DBAPI 2 database connection
pooling
Constructor Arguments
DatabaseConnector

event – Cross-greenthread primitive
greenpool – Green Thread Pools
greenthread – Green Thread Implementation
pools - Generic pools of resources
queue – Queue class
semaphore – Semaphore classes
timeout – Universal Timeouts
websocket – Websocket Server
wsgi – WSGI server
SSL
Non-Standard Extension to Support Post Hooks

eventlet.green.zmq – ØMQ support
zmq
– The pyzmq ØMQ python bindings

bluefire1991
2013/10/27

21:31