pythongevent教程_Geventtutorial(Gevent中⽂教程,Geve。
。。
gevent For the Working Python Developer
Written by the Gevent Community
gevent is a concurrency library bad around libev. It provides a clean API for a variety of concurrency and network related tasks.
gevent是⼀个并发性库基于libev,它提供了⼀个纯净的API ⽤来处理各类问题和⽹络相关任务.
Introduction 介绍
The structure of this tutorial assumes an intermediate level knowledge of Python but not much el. No knowledge of concurrency is expected. The goal is to give you the tools you need to get going with gevent, help you tame your existing concurrency problems and start writing asynchronous applications today.
这教程的结构假设⼀个中等层次的Python程序员且没有什么并发性的知识.⽬的是给你知道怎么使⽤gevent,帮助你解决并发性的问题,和开始编写异步的应⽤程序.
Contributors 贡献者
蒸菜做法This is a collaborative document published under MIT licen. Have something to add? See a typo? Fork and issue a pull request Github. Any and all contributions are welcome.
Core 核⼼
Greenlets
The primary pattern ud in gevent is the Greenlet, a lightweight coroutine provided to Python as a C extension module. Greenlets all run inside of the OS process for the main program but are scheduled cooperatively. This differs from any of the real parallelism constructs provided by multiprocessing or multithreadinglibraries which do spin process and POSIX threads which are truly parallel.
在gevent中主要使⽤Greenlet,给Python提供⼀个轻量级的协同程序,作为⼀个C的扩展模块.Greenlets主程序运⾏的所有系统进程是合理安排的. 这不同于任何multiprocessing或者multithreadi
ng提供的库和POSIX线程,这是真正的并⾏多处理器或多线程库提供真正的并⾏结构。(真难翻译)
Synchronous & Asynchronous Execution 同步&异步执⾏
The core idea of concurrency is that a larger task can be broken down into a collection of subtasks who operation does not depend on the other tasks and thus can be run asynchronously instead of one at a time synchronously. A switch between the two executions is known as a context switch.
A context switch in gevent done through yielding. In this ca example we have two contexts which yield to each other through invoking gevent.sleep(0).
并发的核⼼思想是⼀个更⼤的任务可以分解成多个⼦任务,其运⾏不依赖于其他任务的集合,因此可以异步运⾏ ,⽽不是⼀个在时间 同步。两个执⾏程序间的转换是⼀个关联转换。
在gevent中⼀个关联转换可以通过 yielding来实现.在这个例⼦,两个程序的转换是通过调⽤ gevent.sleep(0).
import gevent
def foo():
print('Running in foo')做冰淇淋
gevent.sleep(0)
print('Explicit context switch to foo again')
def bar():
print('Explicit context to bar')
gevent.sleep(0)
print('Implicit context switch back to bar')
gevent.joinall([
gevent.spawn(foo),
gevent.spawn(bar),
])
Running in foo
Explicit context to bar
Explicit context switch to foo again
Implicit context switch back to bar
It is illuminating to visualize the control flow of the program or walk through it with a debugger to e the context switches as they occur.
在调解器⾥⾯清楚地看到程序在两个转换之间是怎么运⾏的.
The real power of gevent comes when we u it for network and IO bound functions which can be cooperatively scheduled. Gevent has taken care of all the details to ensure that your network libraries will implicitly yield their greenlet contexts whenever possible. I cannot stress enough what a powerful idiom this is. But maybe an example will illustrate.
gevent真正的能⼒在于我们把它⽤于⽹络和IO相关的功能会很好的合作安排.
超搞笑笑话import time
import gevent
from gevent import lect
start = time.time()
tic = lambda: 'at %1.1f conds' % (time.time() - start)
def gr1():
# Busy waits for a cond, but we don't want to
print('Started Polling: ', tic())
lect.lect([], [], [], 2)
print('Ended Polling: ', tic())
def gr2():
# Busy waits for a cond, but we don't want to
print('Started Polling: ', tic())
lect.lect([], [], [], 2)
print('Ended Polling: ', tic())
def gr3():
print("Hey lets do some stuff while the greenlets poll, at", tic())
gevent.sleep(1)
gevent.joinall([
gevent.spawn(gr1),
gevent.spawn(gr2),
gevent.spawn(gr3),
])
Started Polling: at 0.0 conds
Started Polling: at 0.0 conds
Hey lets do some stuff while the greenlets poll, at at 0.0 conds
Ended Polling: at 2.0 conds
Ended Polling: at 2.0 conds
A somewhat synthetic example defines a task function which is non-deterministic (i.e. its output is not guaranteed to give the same result for the same inputs). In this ca the side effect of running the function is that the task paus its execution for a random number of conds.
⼀个⽐较综合的例⼦,定义⼀个task函数,它是不确定的(并不能保证相同的输⼊输出).在这种情况运⾏task函数的作⽤只是暂停其执⾏⼏秒钟的随机数.
import gevent
import random宝宝几个月会坐
淬炼怎么读
促进是什么意思def task(pid):
"""
Some non-deterministic task
"""
gevent.sleep(random.randint(0,2)*0.001)
print('Task', pid, 'done')
def synchronous():
for i in range(1,10):
task(i)
def asynchronous():
threads = [gevent.spawn(task, i) for i in xrange(10)]
gevent.joinall(threads)
print('Synchronous:')
synchronous()
print('Asynchronous:')
asynchronous()
Synchronous:
Task 1 done
Task 2 done
Task 3 done
Task 4 done
Task 5 done
Task 6 done
Task 7 done
Task 8 done
Task 9 done
Asynchronous:
Task 1 done
Task 6 done
Task 5 done
Task 0 done
Task 9 done
Task 8 done
Task 7 done
Task 4 done
Task 3 done
Task 2 done案件剖析报告
In the synchronous ca all the tasks are run quentially, which results in the main programming blocking ( i.e. pausing the execution of the main program ) while each task executes.
在同步的情况所有任务都会顺序的运⾏,当每个任务执⾏的时候导致主程序 blocking.
The important parts of the program are the gevent.spawn which wraps up the given function inside of a Greenlet thread. The list of initialized greenlets are stored in the array threads which is pasd to the gevent.joinall function which blocks the current program to run all the given greenlets. The execution will step forward only when all the greenlets terminate.
程序重要的部分是包装起来的函数gevent.spawn , 它是Greenlet的线程. 初始化的greenlets储存在⼀个数组threads ,然后提交
给 gevent.joinall函数,然后阻塞当前的程序去运⾏所有greenlets.只有当所有greenlets停⽌的时候程序才会继续运⾏.
The important fact to notice is that the order of execution in the async ca is esntially random and that the total execution time in the async ca is much less than the sync ca. In fact the maximum time for the synchronous ca to complete is when each tasks paus for 2 conds resulting in a 20 conds for the whole queue. In the async ca the maximum runtime is roughly 2 conds since none of the tasks block the execution of the others.
要注意的是异步的情况程序是⽆序的,异步的执⾏时间是远少于同步的.事实上同步去完成每个任务停⽌2秒的话,结果是要20秒才能完成整个队列.在异步的情况最⼤的运⾏时间⼤概就是2秒,因为每个任务的执⾏都不会阻塞其他的任务.
A more common u ca, fetching data from a rver asynchronously, the runtime of fetch() will differ between requests given the load on the remote rver.
⼀个更常见的情况,是从服务器上异步获取数据,请求之间 fetch() 的运⾏时间会给服务器带来不同的负载.
童言稚语key
import gevent
import urllib2
import simplejson as json
def fetch(pid):
respon = urllib2.urlopen('/time.json')
result = ad()
json_result = json.loads(result)
datetime = json_result['datetime']
print 'Process ', pid, datetime
return json_result['datetime']
def synchronous():
for i in range(1,10):
fetch(i)
def asynchronous():
threads = []
for i in range(1,10):
threads.append(gevent.spawn(fetch, i))
gevent.joinall(threads)
print 'Synchronous:'
synchronous()
print 'Asynchronous:'
asynchronous()
Determinism 确定性
As mentioned previously, greenlets are deterministic. Given the same inputs and they always produce the same output. For example lets spread a task across a multiprocessing pool compared to a gevent pool.
正如之前提到的,greenlets是确定性的.给相同的输⼊就总会提供相同的输出.例如展开⼀个任务来⽐较⼀个multiprocessing pool和⼀个gevent pool.
import time
def echo(i):
time.sleep(0.001)
return i
# Non Deterministic Process Pool