Tornado Celery Redis实现异步

摘要: Tornado Celery Redis实现异步执行

Celery中两个基本的概念：Broker、Backend

Broker ：其实就是一开始说的 消息队列 ，用来发送和接受消息。

Broker有几个方案可供选择：RabbitMQ，Redis，数据库等

Backend ： 作用是保存结果和状态，可以是Database backend，也可以是Cache backend

Backend有几个方案可供选择：http://docs.celeryproject.org/en/latest/configuration.html#celery-result-backend

安装：

pip install celery
pip install tornado-redis
yum install redis

示例：

tasks.py

from time import sleep
from celery import Celery

backend = 'redis://127.0.0.1:6379/0'
broker = 'redis://127.0.0.1:6379/1'

app = Celery('tasks', backend=backend, broker=broker)

@app.task
def add(x, y):
sleep(10)
return x + y

运行服务：

celery -A tasks worker --loglevel=info

测试：

>>> from tasks import add
>>> r = add.delay(4, 4)
>>> r.ready() # 10s内执行，会输出False，因为add中sleep了10s，10s之后返回True
>>>False
>>> r.status #任务状态
'PENDING'
>>> r.result  # 输出你的hostname
>>>8
>>> r.status
'SUCCESS'

Tronado：

from concurrent.futures import ThreadPoolExecutor

class SleepHandler(tornado.web.RequestHandler):
executor = ThreadPoolExecutor(2)
@tornado.web.asynchronous
@tornado.gen.coroutine
def get(self):
res = yield self.sleep(10,10)
self.finish()

@run_on_executor
def sleep(x,y):
time.sleep(5)
return x+y