Documentation for pulsar 0.9.2. For development docs, go here.

Greenlet Support

Greenlet support facilitates the integration of synchronous third-party libraries into pulsar asynchronous framework. It requires the greenlet library.

If you want to understand how integration works but you are unfamiliar with greenlets, check out the greenlet documentation first. On the other hand, if you need to use it in the context of asynchronous psycopg2 connections for example, you can skip the implementation details.

This application does not use monkey patching and therefore it works quite differently from implicit asynchronous libraries such as gevent. All it does, it provides the user with a limited set of utilities for explicitly transferring execution from one greenlet to a another which execute the blocking call in a greenlet-friendly way.

The caller has the responsibility that the blocking call is greenlet-friendly, i.e. it transfers the control of execution back to the parent greenlet when needed.

Usage

Lets assume you are building an application which uses pulsar asynchronous engine and would like to

  • either use an external library written in blocking style, i.e. without yielding control to the event loop when IO calls are performed.
  • or write your client code without dealing with Future or coroutines, in other words in an implicit asynchronous style. In this way your client code can be used on other frameworks just as well.

In both cases, the greenio application is what you need.

Green WSGI

Assume you are using pulsar web server and would like to write your application in an implicit asynchronous mode, i.e. without dealing with futures nor coroutines, then you can wrap your wsgi app with the RunInPool utility:

from pulsar.apps import wsgi, greenio

callable = wsgi.WsgiHandler([wsgi.wait_for_body_middleware,
                             greenio.RunInPool(app, 20)])

wsgi.WsgiServer(callable=callable).start()

The RunInPool manages a pool of greenlets which execute your application. In this way, within your app you can invoke the wait() function when needing to wait for asynchronous results to be ready.

Green Http

The HttpClient can be used with greenlets:

>>> from pulsar.apps.http import HttpClient
>>> http = HttpClient(green=True)
>>> http.green
True

And now you can write synchronous looking code and run it in a separate greenlet via the run_in_greenlet() decorator:

@greenio.run_in_greenlet
def example():
    response = http.get('http://bbc.co.uk')
    ...
    return 'done'

and somewhere, in your asynchronous code:

result = yield example()
result == 'done'

the run_in_greenlet() decorator, execute the function on a child greenlet without blocking the asynchronous engine. Once the example function returns, the asynchronous code continue from the yield statement as usual.

API

Wait

pulsar.apps.greenio.wait(coro_or_future, loop=None)[source]

Wait for a coroutine or a Future to complete.

This function must be called from a greenlet other than the main one. It can be used in conjunction with the run_in_greenlet() decorator or the GreenPool.

Wait file descriptor

pulsar.apps.greenio.wait_fd(fd, read=True)[source]

Wait for an event on file descriptor fd.

Parameters:
  • fd – file descriptor
  • read=True – wait for a read event if True, otherwise a wait for write event.

This function must be invoked from a coroutine with parent, therefore invoking it from the main greenlet will raise an exception. Check how this function is used in the psycopg2_wait_callback() function.

Run in greenlet

pulsar.apps.greenio.run_in_greenlet(callable)[source]

Decorator to run a callable on a new greenlet.

A callable decorated with this decorator returns a coroutine

Green task

pulsar.apps.greenio.green_task(method)[source]

Decorator to run a method an a new greenlet in the event loop of the instance of the bound method.

This method is the greenlet equivalent of the task() decorator. The instance must be an async object.

Returns:a Future

Green Pool

class pulsar.apps.greenio.pool.GreenPool(max_workers=None, loop=None, maxtasks=None)[source]

A pool of running greenlets.

This pool maintains a group of greenlets to perform asynchronous tasks via the submit() method.

submit(func, *args, **kwargs)[source]

Equivalent to func(*args, **kwargs).

This method create a new task for function func and adds it to the queue. Return a Future called back once the task has finished.

Run in Pool

class pulsar.apps.greenio.pool.RunInPool(app, max_workers=None, loop=None)[source]

Utility for running a callable in a GreenPool.

Parameters:
  • app – the callable to run on greenlet workers
  • max_workers=100 – maximum number of workers
  • loop – optional event loop

THis utility is used by the pulse application.

Psycopg2

Pulsar greenio application can be used in conjunction with psycopg2 coroutine support to query the PostgreSql database in an implicit asynchronous fashion. Such feature can be extremely useful since it allows to write asynchronous code without the need to explicitly yield control appropriately when necessary.

For example, one could use ORMs (Object Relational Mappers) such as SqlAlchemy and django in asynchronous mode by invoking make_asynchronous() somehere convinient in the code:

from pulsar.apps.greenio import pg

pg.make_asynchronous()

The make_asynchronous() set the psycopg2_wait_callback() function as the waiting callback in psycopg2.

See also

Check out how the pulse application uses greenlet support to asynchrnously execute django middleware when using PostgreSql as backend database.

pulsar.apps.greenio.pg.psycopg2_wait_callback(conn)[source]

A wait callback to allow greenlet to work with Psycopg. The caller must be from a greenlet other than the main one.

pulsar.apps.greenio.pg.make_asynchronous()[source]


Table Of Contents

Previous topic

Django Application

Next topic

Distributed Task Queue

This Page