Flask源码解读(1) -- app.run()的背后

Flask源码解读(1) -- app.run()的背后

本系列文章分析用python实现的web后端框架Flask, 分析其运行, session, routing, context等.
当我们用Flask写好一个app后, 运行app.run()表示监听指定的端口, 对收到的request运行app生成response并返回. 现在分析一下, 运行app.run()后具体发生了什么事情Flask定义的run方法如下:

def run(self, host=None, port=None, debug=None, load_dotenv=True, **options):
# Change this into a no-op if the server is invoked from the
# command line.  Have a look at cli.py for more information.
if os.environ.get('FLASK_RUN_FROM_CLI') == 'true':
from .debughelpers import explain_ignored_app_run
explain_ignored_app_run()
return

if load_dotenv:
from flask.cli import load_dotenv
load_dotenv()

if debug is not None:
self._reconfigure_for_run_debug(bool(debug))

_host = '127.0.0.1'
_port = 5000
server_name = self.config.get("SERVER_NAME")
sn_host, sn_port = None, None

if server_name:
sn_host, _, sn_port = server_name.partition(':')

host = host or sn_host or _host
port = int(port or sn_port or _port)
options.setdefault('use_reloader', self.debug)
options.setdefault('use_debugger', self.debug)

from werkzeug.serving import run_simple

try:
run_simple(host, port, self, **options)
finally:
# reset the first request information if the development server
# reset normally.  This makes it possible to restart the server
# without reloader and that stuff from an interactive shell.
self._got_first_request = False

在上面的源码中, from werkzeug.serving import run_simple引入run_simple函数,  该函数将定义好的app在指定的host, port上运行  

在上面的源码中, from werkzeug.serving import run_simple引入run_simple函数,  该函数将定义好的app在指定的host, port上运行

werkzeug.serving 中的 run_simple函数如下:

def run_simple(hostname, port, application, use_reloader=False,
extra_files=None, threaded=False, processes=1):
def inner():
srv = make_server(hostname, port, application, threaded,
processes)
try:
srv.serve_forever()
except KeyboardInterrupt:
pass

if os.environ.get('WERKZEUG_RUN_MAIN') != 'true':
print '* Running on http://%s:%d/' % (hostname or '0.0.0.0', port)
if use_reloader:
test_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
test_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
test_socket.bind((hostname, port))
test_socket.close()
run_with_reloader(inner, extra_files or [])
else:
inner()

一般情况下,先定义inner函数再调用他. inner函数通过make_server函数生成一个srv, 再调用srv.serve_forevermake_server函数:

def make_server(host, port, app=None, threaded=False, processes=1):

if threaded and processes > 1:
raise ValueError("cannot have a multithreaded and "
"multi process server.")
elif threaded:
class handler(BaseRequestHandler):
multithreaded = True
class server(ThreadingMixIn, WSGIServer):
pass
elif processes > 1:
class handler(BaseRequestHandler):
multiprocess = True
max_children = processes - 1
class server(ForkingMixIn, WSGIServer):
pass
else:
handler = BaseRequestHandler
server = WSGIServer
srv = server((host, port), handler)
srv.set_app(app)
return srv

make_server函数通过指定的参数,决定后面要用到的handler类和server类的类型. 参数可指定为多线程, 多进程或者默认形式
可以看到在指定为多线程或多线程的时候, handler类的类属性multithreaded或者multiprocess被设置, server类则通过继承自不同的类来区别.
确定了hanlder, server类之后, 还通过srv.set_app(app)将app绑定到srv中再返回srv.
下面我们先来看server的基础类WSGIServer, WSGIServer类的引入如下:
from wsgiref.simple_server import ServerHandler, WSGIRequestHandler, WSGIServer
from SocketServer import ThreadingMixIn, ForkingMixIn
可以看到上面的类, werkzeug是从别的模块引入的
wsgiref.simple_server中的WSGIServer如下:

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
class WSGIServer(HTTPServer):

application = None

def server_bind(self):
HTTPServer.server_bind(self)
self.setup_environ()

def setup_environ(self):
# Set up base environment
env = self.base_environ = {}
env['SERVER_NAME'] = self.server_name
env['GATEWAY_INTERFACE'] = 'CGI/1.1'
env['SERVER_PORT'] = str(self.server_port)
env['REMOTE_HOST']=''
env['CONTENT_LENGTH']=''
env['SCRIPT_NAME'] = ''

def get_app(self):
return self.application

def set_app(self,application):
self.application = application

可以看到WSGIServer继承自来自BaseHTTPServer模块的HTTPServer. 
class HTTPServer(SocketServer.TCPServer): HTTPServer继承自来自SocketServer模块的TCPServer
class TCPServer(BaseServer): 在SocketServer模块中, TCPServer继承自BaseServer
BaseServer中定义了前面出现的serve_forever方法

class BaseServer:
def serve_forever(self, poll_interval=0.5):
self.__is_shut_down.clear()

try:

with _ServerSelector() as selector:
selector.register(self, selectors.EVENT_READ)

while not self.__shutdown_request:

b372
ready = selector.select(poll_interval)
if ready:
self._handle_request_noblock()

self.service_actions()
finally:
self.__shutdown_request = False
self.__is_shut_down.set()

selector是监听多个文件描述符的对象. serve_forever函数先注册自己绑定地址上的EVENT_READ事件, 当selector.select
函数返回时, 表示有一个连接进来. 此时调用self._handle_request_noblock方法

class BaseServer:
def _handle_request_noblock(self):

try:
request, client_address = self.get_request()
except OSError:
return
if self.verify_request(request, client_address):
try:
self.process_request(request, client_address)
except Exception:
self.handle_error(request, client_address)
self.shutdown_request(request)
except:
self.shutdown_request(request)
raise
else:
self.shutdown_request(request)

可以看到在self._handle_request_noblock方法中, 先调用self.get_request方法. self.get_request方法
其实是调用了监听socket的accept方法. 该方法表示在tcp协议下, 服务器端接受一个新连接, 函数的返回值为一个文件描述符
注意建立的连接实际是由客户端和返回的新文件描述符构成, 监听socket不变, 继续监听其他的连接请求
回到request, client_address = self.get_request()语句, 返回的request表示新建的文件描述符, 可以在request上读写.
client_address表示客户端的地址. 如果该步骤执行成功, 连接就建立好了, 后面开始正式的处理请求.
后面的代码, 先通过self.verify_request验证请求, 再通过self.process_request处理请求. 读者可以看看代码中
对异常情况的处理. 在建立连接-验证连接-处理连接整个过程中, 都有可能会出现异常, 缜密的考虑异常处理值得我们学习.
下面我们重点看self.process_request方法, 这是处理请求的核心. 
self.process_request方法调用了self.finish_request方法, self.finish_request如下:

class BaseServer:
def finish_request(self, request, client_address):
self.RequestHandlerClass(request, client_address, self)

发现建立了一个self.RequestHandlerClass类的对象. self.RequestHandlerClass属性在建立server对象的时候指定
回顾一下server实例建立的过程, Flask中的run -> werkzeug.serving.run_simple -> werkzeug.serving.make_server
在make_server中确定了hanlder类和server类, 再通过srv = server((host, port), handler)建立了srv, 可以发现
在创建server对象的时候, 传入了handler类做为参数, 这个handler就是上面提到过的self.RequestHandlerClass属性
在make_server函数中handler类是BaseRequestHandler, 这个类又通过一系列的继承(跟上面server类差不多),追溯到
SocketServer中的BaseRequestHandler类(这里出现了名称冲突, 我将make_server函数指向的BaseRequestHandler
改名为BaseRequestHandlerWerkzeug). SocketServer中的BaseRequestHandler中定义了__init__方法

class BaseRequestHandler:
def __init__(self, request, client_address, server):
self.request = request
self.client_address = client_address
self.server = server
self.setup()
try:
self.handle()
finally:
self.finish()

__init__方法中, 先绑定了前面server实例中的request(可读可写的文件描述符), client_address(客户端地址), server本身. 然后调用setup方法建立环境, 在BaseRequestHandler中没有定义setup方法, 由子类实现.
之后调用handle方法, 同样在BaseRequestHandler中没有定义handle方法, 由子类实现. 重点看handle方法

在BaseRequestHandlerWerkzeug中定义了handle方法

class BaseRequestHandlerWerkzeug(WSGIRequestHandler):

multithreaded = False
multiprocess = False
_handler_class = None

def get_handler(self):
handler = self._handler_class
if handler is None:
class handler(ServerHandler):
wsgi_multithread = self.multithreaded
wsgi_multiprocess = self.multiprocess
self._handler_class = handler

rv = handler(self.rfile, self.wfile, self.get_stderr(),
self.get_environ())
rv.request_handler = self
return rv

def handle(self):
self.raw_requestline = self.rfile.readline()
if self.parse_request():
self.get_handler().run(self.server.get_app())

handle方法中, 先调用self.rfile.readline读取一行, self.rfile是前面self.request的读端. 读取的一行值存入到self.raw_requestline属性中, 熟悉http协议的读者应该知道, 这是每个请求的起始行, 包含请求方法, url, http版本. 后面的self.parse_request方法就是分析此起始行并添加了self.command, self.path, self.request_version属性. 
再看看self.get_handler方法, 先设定好handler类, 默认情况下继承自wsgiref.simple_server模块的ServerHandler类, 生成handler类实例rv时, 四个参数分别为self.rfile(表示连接的文件描述符的读端), self.wfile(写端), self.get_stderr()(错误输出处), self.get_environ()(生成了一个environ对向)

class WSGIRequestHandler(BaseHTTPRequestHandler):

def get_environ(self):
env = self.server.base_environ.copy()
env['SERVER_PROTOCOL'] = self.request_version
env['REQUEST_METHOD'] = self.command
if '?' in self.path:
path,query = self.path.split('?',1)
else:
path,query = self.path,''

env['PATH_INFO'] = urllib.unquote(path)
env['QUERY_STRING'] = query

host = self.address_string()
if host != self.client_address[0]:
env['REMOTE_HOST'] = host
env['REMOTE_ADDR'] = self.client_address[0]

if self.headers.typeheader is None:
env['CONTENT_TYPE'] = self.headers.type
else:
env['CONTENT_TYPE'] = self.headers.typeheader

length = self.headers.getheader('content-length')
if length:
env['CONTENT_LENGTH'] = length

for h in self.headers.headers:
k,v = h.split(':',1)
k=k.replace('-','_').upper(); v=v.strip()
if k in env:
continue                    # skip content length, type,etc.
if 'HTTP_'+k in env:
env['HTTP_'+k] += ','+v     # comma-separate multiple headers
else:
env['HTTP_'+k] = v
return env

这个env对象便是传给我们编写好的app的环境, app根据env生成response. 在handler类中没有定义run方法, 需要到父类中寻找. 
下面我们来看ServerHandler类, 又是一系列复杂的类继承, ServerHandler <- SimpleHandler <- BaseHandler.在SimpleHandler类中定义了self.__init__方法

class SimpleHandler(BaseHandler):

def __init__(self,stdin,stdout,stderr,environ,
multithread=True, multiprocess=False
):
self.stdin = stdin
self.stdout = stdout
self.stderr = stderr
self.base_env = environ
self.wsgi_multithread = multithread
self.wsgi_multiprocess = multiprocess

这个__init__方法比较简单, 就是各种绑定在BaseHandler中定义了self.run方法

class BaseHandler:

def run(self, application):
try:
self.setup_environ()
self.result = application(self.environ, self.start_response)
self.finish_response()
except:
try:
self.handle_error()
except:
# If we get an error handling an error, just give up already!
self.close()
raise   # ...and let the actual server figure it out.

可以看到先调用self.setup_environ方法为前面的env对象添加另外一些environ键值对, 再通过self.result = application(self.environ, self.start_response)得到结果. 现在我们知道了app必须是定义了__call__方法的对象, 这样在application(self.environ, self.start_response)时, 向__call__方法传入两个参数得到结果而在Flask源码中, 确实为app定义了__call__方法, 这也是整个app处理请求的起点.

总结:

调用app.run背后, 流程可分为两大块. 一块是建立server对象, 负责和客户段建立连接的逻辑, 决定采用TCP/UDP, 是否参用多线程/多进程等另一块建立handler对象, 负责对从建立好的连接中获得的请求的处理, handler将请求打包成environ对象, 在Flask框架下编写的app据此environ生成response