Openstack源代码分析之PasteDeploy+Webob实例以及Openstack源代码下PasteDeploy+Webob+Routes分析

以下内容转自 ：http://blog.csdn.net/bluefire1991/article/details/13614243 作者：bluefire1991

通过PasteDeploy+Webob来配置WSGI服务器接口

Webob是一种封装了HTTP协议的模块，具体课参考官方文档，不过这两天不知为什么不能访问，我是直接下载的源代码，源代码下docs自带本地文档，可以通过sphnix-builder的命令来生成本地文档

测试了两种方案

一种是不使用Webob装饰器的方式

一种是使用Webob装饰器的方式

配置文件如下test-deploy.ini

[plain] view
plaincopy

[DEFAULT]

key1=value1

key2=value2

key3=values

[composite:main]

use=egg:Paste#urlmap

/=show

/auther=auther

/version=version

[pipeline:show]

pipeline = auth root

[pipeline:version]

pipeline = logrequest showversion

[pipeline:auther]

pipeline = logrequest showauther

[filter:logrequest]

username = root

password = 123

paste.filter_factory = test.testdeploy:log_factory

[app:showversion]

version = 1.0.0

paste.app_factory = test.testdeploy:version_factory

[app:showauther]

auther = bluefire1991

paste.app_factory = test.testdeploy:showauther_factory

[app:root]

paste.app_factory = test.testdeploy:show_factory

[filter:auth]

paste.filter_factory = test.testdeploy:filter_factory

配置方案用的类似openstack实现的配置方案，paste.xxx_factory和pipeline的方式实现配置，不过路径的配置openstack源代码实现的是用Routes实现RESTful的功能，这里没有用Routes直接在ini文件下配置的路径。为什么这样配置可以参考我的上一篇博客。

代码文件testdeploy.py

[python] view
plaincopy

'''''

Created on 2013-10-28

@author: root

'''

import logging

import os

import sys

import webob

from webob import Request

from webob import Response

from webob.dec import *

from webob.exc import *

from paste.deploy import loadapp

from wsgiref.simple_server import make_server

import signal

def sigint_handler(signal, frame):

"""Exits at SIGINT signal."""

logging.debug('SIGINT received, stopping servers.')

sys.exit(0)

#用于封装app

@wsgify

def test(request):

return Response('Here!')

#用于封装app

@wsgify

def application(request):

return Response('Hello and Welcome!')

#wsgify.middleware用于wsgi的对外filter层，必须要两个参数，request对象和app对象，app用于向下一个filter或者app传递参数

@wsgify.middleware

def auth_filter(request, app):

if request.headers.get('X-Auth-Token') != 'bluefire1991':

#类似于在这里写了start_response和return函数，只不过这里由webob的Request对象封装好了，本来test是一个函数对象，调用需要test(req),

#通过装饰器@wsgi直接变成一个对象，参数在装饰器内部实现wsgify(test)

return test

return app(request)

#app_factory

def show_factory(global_conf,**local_conf):

return application

#app_factory

def version_factory(global_conf,**local_conf):

return ShowVersion(global_conf,local_conf)

#app_fatory

def showauther_factory(global_conf,**local_conf):

return ShowAuther(global_conf,local_conf)

#filter_factory

def filter_factory(global_conf, **local_conf):

return auth_filter

#filter_factory

def log_factory(global_conf,**local_conf):

def filter(app):

return LogFilter(app,global_conf,local_conf)

return filter

class LogFilter():

def __init__(self,app,global_conf,local_conf):

self.app = app

self.global_conf=global_conf

self.local_conf=local_conf

def __call__(self,environ,start_response):

print "filter:LogFilter is called."

req = Request(environ)

if req.GET.get("username", "")==self.local_conf['username'] and req.GET.get("password", "")==self.local_conf['password']:

return self.app(environ,start_response)

start_response("200 OK",[("Content-type", "text/plain")])

return ["You are not authorized"]

class ShowVersion():

def __init__(self,global_conf,local_conf):

self.global_conf=global_conf

self.local_conf=local_conf

def __call__(self,environ,start_response):

start_response("200 OK",[("Content-type", "text/plain")])

return ['Version',self.local_conf['version']]

class ShowAuther():

def __init__(self,global_conf,local_conf):

self.global_conf=global_conf

self.local_conf=local_conf

def __call__(self,environ,start_response):

res = Response()

res.status = "200 OK"

res.content_type = "text/plain"

# get operands

res.body = ["auther",self.local_conf['auther']]

return res

if __name__ == '__main__':

signal.signal(signal.SIGINT, sigint_handler)

configfile="test-deploy.ini"

appname="main"

wsgi_app = loadapp("config:%s" % os.path.abspath(configfile), appname)

server = make_server('localhost',8088,wsgi_app)

server.serve_forever()

pass

由代码和配置文件分析，配置文件有三个路径，/,/auther,/version,/的实现路径是filter_factory（auth_filter）->show_factory（application）,/auther和/version的实现方案是由log_factory（LogFilter）->showauther_factory（ShowAuther.__call___）和version_factory（ShowVersion.__call__）,/路径的方案都是直接通过wsgi的装饰器实现的，/verison是python原生的start_response方案实现的，/auther是利用webob封装的Response对象实现的。

但是到openstack这里有一个问题，在keystone处理时，有多个路径，不可能都放在ini里面配置（而且里面也没有），那这么多路径他是怎么实现的呢？

源代码说明，他用了Routes，来实现路径映射功能，官方文档地址是Routes官方文档

先放几段Openstack的关键源代码，他在xxx_factory（都很类似）实现的代码如下

[python] view
plaincopy

@fail_gracefully

def public_app_factory(global_conf, **local_conf):

controllers.register_version('v2.0')

conf = global_conf.copy()

conf.update(local_conf)

return wsgi.ComposingRouter(routes.Mapper(),

[identity.routers.Public(),

token.routers.Router(),

routers.VersionV2('public'),

routers.Extension(False)])

实现了一个ComposingRouter对象，其中router.Mapper()基于创建了一个路由对象，通过这个就可以实现映射，ComposingRoute类实现

[python] view
plaincopy

class ComposingRouter(Router):

def __init__(self, mapper=None, routers=None):

if mapper is None:

mapper = routes.Mapper()

if routers is None:

routers = []

for router in routers:

router.add_routes(mapper)

super(ComposingRouter, self).__init__(mapper)

可见创建了mapper对象，调用[]里面所有对象（identity.routers.Public())下的add_routers方法，再跟进add_routers方法看看，以identity.routers.Public()为例

[python] view
plaincopy

class Public(wsgi.ComposableRouter):

def add_routes(self, mapper):

tenant_controller = controllers.Tenant()

mapper.connect('/tenants',

controller=tenant_controller,

action='get_projects_for_token',

conditions=dict(method=['GET']))

可见，他用mapper.connect生成了一个路径/tenants,访问方式必须为GET，调用函数为tenant_controller(controller.Tenant()对象)下get_projects_for_token函数。如下

[python] view
plaincopy

class Tenant(controller.V2Controller):

def get_all_projects(self, context, **kw):

"""Gets a list of all tenants for an admin user."""

if 'name' in context['query_string']:

return self.get_project_by_name(

context, context['query_string'].get('name'))

self.assert_admin(context)

tenant_refs = self.identity_api.list_projects()

for tenant_ref in tenant_refs:

tenant_ref = self.filter_domain_id(tenant_ref)

params = {

'limit': context['query_string'].get('limit'),

'marker': context['query_string'].get('marker'),

}

return self._format_project_list(tenant_refs, **params)

def get_projects_for_token(self, context, **kw):

"""Get valid tenants for token based on token used to authenticate.

Pulls the token from the context, validates it and gets the valid

tenants for the user in the token.

Doesn't care about token scopedness.

"""

try:

token_ref = self.token_api.get_token(context['token_id'])

except exception.NotFound as e:

LOG.warning('Authentication failed: %s' % e)

raise exception.Unauthorized(e)

user_ref = token_ref['user']

tenant_refs = (

self.assignment_api.list_projects_for_user(user_ref['id']))

tenant_refs = [self.filter_domain_id(ref) for ref in tenant_refs

if ref['domain_id'] == DEFAULT_DOMAIN_ID]

params = {

'limit': context['query_string'].get('limit'),

'marker': context['query_string'].get('marker'),

}

return self._format_project_list(tenant_refs, **params)

这时又有两个疑问产生了：

1.paste_factory的对象不是用webob来封装的HTTP服务吗，我在get_projects_for_token函数下没看见啊？

2.mapper.connect()配置完成后为什么就能实现映射了呢？

问题1.我的理解：

Tenant的基类是V2Controller，V2Controller的基类是class V2Controller(wsgi.Application)，而在wsgi.Application类中实现如下

[python] view
plaincopy

class Application(BaseApplication):

@webob.dec.wsgify(RequestClass=Request)

def __call__(self, req):

arg_dict = req.environ['wsgiorg.routing_args'][1]

action = arg_dict.pop('action')

del arg_dict['controller']

LOG.debug(_('arg_dict: %s'), arg_dict)

# allow middleware up the stack to provide context, params and headers.

context = req.environ.get(CONTEXT_ENV, {})

context['query_string'] = dict(req.params.iteritems())

context['headers'] = dict(req.headers.iteritems())

context['path'] = req.environ['PATH_INFO']

params = req.environ.get(PARAMS_ENV, {})

for name in ['REMOTE_USER', 'AUTH_TYPE']:

try:

context[name] = req.environ[name]

except KeyError:

try:

del context[name]

except KeyError:

pass

params.update(arg_dict)

context.setdefault('is_admin', False)

# TODO(termie): do some basic normalization on methods

method = getattr(self, action)

# NOTE(vish): make sure we have no unicode keys for py2.6.

params = self._normalize_dict(params)

try:

result = method(context, **params)

except exception.Unauthorized as e:

LOG.warning(

_('Authorization failed. %(exception)s from %(remote_addr)s') %

{'exception': e, 'remote_addr': req.environ['REMOTE_ADDR']})

return render_exception(e, user_locale=req.best_match_language())

except exception.Error as e:

LOG.warning(e)

return render_exception(e, user_locale=req.best_match_language())

except TypeError as e:

LOG.exception(e)

return render_exception(exception.ValidationError(e),

user_locale=req.best_match_language())

except Exception as e:

LOG.exception(e)

return render_exception(exception.UnexpectedError(exception=e),

user_locale=req.best_match_language())

if result is None:

return render_response(status=(204, 'No Content'))

elif isinstance(result, basestring):

return result

elif isinstance(result, webob.Response):

return result

elif isinstance(result, webob.exc.WSGIHTTPException):

return result

response_code = self._get_response_code(req)

return render_response(body=result, status=response_code)

终于看到了久违的wsgi了，那我怎么在调用对象时候，调用我需要的函数呢，代码很清楚

[python] view
plaincopy

method = getattr(self, action)

# NOTE(vish): make sure we have no unicode keys for py2.6.

params = self._normalize_dict(params)

try:

result = method(context, **params)

getattr里action就是我们上面的需要的函数，在这里调用，再通过render_response(body=result, status=response_code)，render_response是自己的http返回函数。问题到这里应该能得到解答（？）

问题2，我的理解：

跟到class ComposingRouter(Router)的基类Router

[python] view
plaincopy

class Router(object):

"""WSGI middleware that maps incoming requests to WSGI apps."""

def __init__(self, mapper):

"""Create a router for the given routes.Mapper.

Each route in `mapper` must specify a 'controller', which is a

WSGI app to call. You'll probably want to specify an 'action' as

well and have your controller be an object that can route

the request to the action-specific method.

Examples:

mapper = routes.Mapper()

sc = ServerController()

# Explicit mapping of one route to a controller+action

mapper.connect(None, '/svrlist', controller=sc, action='list')

# Actions are all implicitly defined

mapper.resource('server', 'servers', controller=sc)

# Pointing to an arbitrary WSGI app. You can specify the

# {path_info:.*} parameter so the target app can be handed just that

# section of the URL.

mapper.connect(None, '/v1.0/{path_info:.*}', controller=BlogApp())

"""

# if we're only running in debug, bump routes' internal logging up a

# notch, as it's very spammy

if CONF.debug:

logging.getLogger('routes.middleware')

self.map = mapper

self._router = routes.middleware.RoutesMiddleware(self._dispatch,

self.map)

@webob.dec.wsgify(RequestClass=Request)

def __call__(self, req):

"""Route the incoming request to a controller based on self.map.

If no match, return a 404.

"""

return self._router

@staticmethod

@webob.dec.wsgify(RequestClass=Request)

def _dispatch(req):

"""Dispatch the request to the appropriate controller.

Called by self._router after matching the incoming request to a route

and putting the information into req.environ. Either returns 404

or the routed WSGI app's response.

"""

match = req.environ['wsgiorg.routing_args'][1]

if not match:

return render_exception(

exception.NotFound(_('The resource could not be found.')),

user_locale=req.best_match_language())

app = match['controller']

return app

根据Router基类下routes对象自己的实现机制__dispatch和map实现对请求路径匹配，在找到路径后根据routes机制即可映射wsgi服务（怎么实现的？routes官方文档没有搜索到）。问题2应该能得到解答（？）