Documentation for pulsar 0.9.2. For development docs, go here.
Pulsar ships with a thread safe, fully featured, HttpClient
class for multiple asynchronous HTTP requests.
To get started, one builds a client:
>>> from pulsar.apps import http
>>> client = http.HttpClient()
and than makes requests, in a coroutine:
response = yield http.get('http://www.bbc.co.uk')
or (only in python 3):
response = yield from http.get('http://www.bbc.co.uk')
Pulsar HTTP client has no dependencies and an API similar to requests:
from pulsar.apps import http
client = http.HttpClient()
response = yield client.get('https://github.com/timeline.json')
response
is a HttpResponse
object which contains all the
information about the request and the result:
>>> request = response.request
>>> print(request.headers)
Connection: Keep-Alive
User-Agent: pulsar/0.8.2-beta.1
Accept-Encoding: deflate, gzip
Accept: */*
>>> response.status_code
200
>>> print(response.headers)
...
The request
attribute of HttpResponse
is an instance of HttpRequest
.
Cookies are handled by the client by storing cookies received with responses.
To disable cookie one can pass store_cookies=False
during
HttpClient
initialisation.
If a response contains some Cookies, you can get quick access to them:
>>> response = yield client.get(...)
>>> type(response.cookies)
<type 'dict'>
To send your own cookies to the server, you can use the cookies parameter:
response = client.get(..., cookies={'sessionid': 'test'})
Authentication, either basic
or digest
, can be added to a
client by invoking
In either case the authentication is handled by adding additional headers to your requests.
Supported out of the box:
client = HttpClient()
client.get('https://github.com/timeline.json')
you can include certificate file and key too, either
to a HttpClient
or to a specific request:
client = HttpClient(certkey='public.key')
res1 = client.get('https://github.com/timeline.json')
res2 = client.get('https://github.com/timeline.json',
certkey='another.key')
This is an event-driven client, therefore streaming support is native.
To stream data received from the client one uses the data_processed event handler. For example:
def new_data(response, **kw):
if response.status_code == 200:
data = response.recv_body()
# do something with this data
response = http.get(..., data_processed=new_data)
The response recv_body()
method fetches the parsed body
of the response and at the same time it flushes it.
Check the proxy server example for an
application using the HttpClient
streaming capabilities.
The http client support websocket upgrades. First you need to have a websocket handler:
from pulsar.apps import ws
class Echo(ws.WS):
def on_message(self, websocket, message):
websocket.write(message)
The websocket response is obtained by:
ws = yield http.get('ws://...', websocket_handler=Echo())
[TODO]
Events control the behaviour of the
HttpClient
when certain conditions occur. They are useful for
handling standard HTTP event such as redirects,
websocket upgrades,
streaming or anything your application
requires.
There are three one time events associated with an
HttpResponse
object:
pre_request
, fired before the request is sent to the server. Callbacks
receive the response argument.on_headers
, fired when response headers are available. Callbacks
receive the response argument.post_request
, fired when the response is done. Callbacks
receive the response argument.Adding event handlers can be done at client level:
def myheader_handler(response, exc=None):
if not exc:
print('got headers!')
client.bind_event('on_headers', myheader_handler)
or at request level:
response = client.get(..., on_headers=myheader_handler)
By default, the HttpClient
has one pre_request
callback for
handling HTTP tunneling, three on_headers
callbacks for
handling 100 Continue, websocket upgrade and cookies,
and one post_request
callback for handling redirects.
In addition to the three one time events, the Http client supports two additional events which can occur several times while processing a given response:
data_received
is fired when new data has been received but not yet
parseddata_processed
is fired just after the data has been parsed by the
HttpResponse
. This is the event one should bind to when performing
http streaming.both events support handlers with a signature:
def handler(response, data=None):
...
where response
is the HttpResponse
handling the request and
data
is the raw data received.
The main class here is the HttpClient
which is a subclass of
AbstractClient
.
pulsar.apps.http.
HttpClient
(proxy_info=None, cache=None, headers=None, encode_multipart=True, multipart_boundary=None, keyfile=None, certfile=None, cert_reqs=0, ca_certs=None, cookies=None, store_cookies=True, max_redirects=10, decompress=True, version=None, websocket_handler=None, parser=None, trust_env=True, loop=None, client_version=None, timeout=None, pool_size=10, green=False)[source]¶A client for HTTP/HTTPS servers.
It handles pool of asynchronous connections.
Parameters: |
|
---|
headers
¶Default headers for this HttpClient
.
Default: DEFAULT_HTTP_HEADERS
.
Default cookies for this HttpClient
.
If True
it remebers response cookies and send them back to
serves.
Default: True
timeout
¶Default timeout for the connecting sockets. If 0 it is an asynchronous client.
encode_multipart
¶Flag indicating if body data is by default encoded using the
multipart/form-data
or application/x-www-form-urlencoded
encoding.
It can be overwritten during a request()
.
Default: True
proxy_info
¶Dictionary of proxy servers for this client.
pool_size
¶The size of a pool of connection for a given host.
connection_pools
¶Dictionary of connection pools for different hosts
green
¶If True
the requests are processed using the
greenio
application. When operating in this mode,
the HttpClient
must be used from a greenlet ather than the
main one.
DEFAULT_HTTP_HEADERS
¶Default headers for this HttpClient
client_version
= 'pulsar/0.9.2'¶String for the User-Agent
header.
version
= 'HTTP/1.1'¶Default HTTP request version for this HttpClient
.
It can be overwritten on request()
.
get
(url, **kwargs)[source]¶Sends a GET request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
options
(url, **kwargs)[source]¶Sends a OPTIONS request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
head
(url, **kwargs)[source]¶Sends a HEAD request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
post
(url, **kwargs)[source]¶Sends a POST request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
put
(url, **kwargs)[source]¶Sends a PUT request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
patch
(url, **kwargs)[source]¶Sends a PATCH request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
delete
(url, **kwargs)[source]¶Sends a DELETE request and returns a HttpResponse
object.
Params url: | url for the new HttpRequest object. |
---|---|
Parameters: | **kwargs – Optional arguments for the request() method. |
request
(*args, **kwargs)[source]¶Constructs and sends a request to a remote server.
It returns a Future
which results in a
HttpResponse
object.
Parameters: |
|
---|---|
Return type: | a |
close
(async=True, timeout=5)[source]¶Close all connections.
Fire the finish
one time event once done.
Return the Future
fired by the finish
event.
pulsar.apps.http.
HttpRequest
(client, url, method, inp_params, headers=None, data=None, files=None, timeout=None, history=None, charset=None, encode_multipart=True, multipart_boundary=None, source_address=None, allow_redirects=False, max_redirects=10, decompress=True, version=None, wait_continue=False, websocket_handler=None, cookies=None, **ignored)[source]¶An HttpClient
request for an HTTP resource.
This class has a similar interface to urllib.request.Request
.
Parameters: |
|
---|
method
¶The request method
version
¶HTTP version for this request, usually HTTP/1.1
history
¶List of past HttpResponse
(collected during redirects).
wait_continue
¶if True
, the HttpRequest
includes the
Expect: 100-Continue
header.
address
¶(host, port)
tuple of the HTTP resource
ssl
¶Context for TLS connections.
If this is a tunneled request and the tunnel connection is not yet
established, it returns None
.
proxy
¶Proxy server for this request.
full_url
¶Full url of endpoint
data
¶Body of request
encode
()[source]¶The bytes representation of this HttpRequest
.
Called by HttpResponse
when it needs to encode this
HttpRequest
before sending it to the HTTP resource.
pulsar.apps.http.
HttpResponse
(loop=None, one_time_events=None, many_times_events=None)[source]¶A ProtocolConsumer
for the HTTP client protocol.
Initialised by a call to the HttpClient.request
method.
There are two events you can yield in a coroutine:
on_headers
¶fired once the response headers are received.
on_finished
¶Fired once the whole request has finished
Public API:
status_code
¶Numeric status code such as 200, 404 and so forth.
Available once the on_headers
has fired.
url
¶The request full url.
Dictionary of cookies set by the server or None
.
pulsar.apps.http.
OAuth1
(client_id=None, client=None, **kw)[source]¶Add OAuth1 authentication to pulsar HttpClient
pulsar.apps.http.
OAuth2
(client_id=None, client=None, **kw)[source]¶Add OAuth2 authentication to pulsar HttpClient