Documentation for pulsar 0.9.2. For development docs, go here.
Documentation for utilities used by pulsar internals. This module is independent from all other pulsar modules and therefore can be used as a stand-alone library.
This is a substantial module which imporfts several classes and functions
from the standard library in a python 2.6 to python 3.3 compatible fashion.
On top of that, it implements the HttpClient
for handling synchronous
and asynchronous HTTP requests in a pythonic way.
It is a thin layer on top of urllib2 in python2 / urllib in Python 3. Several opensource efforts have been used as source of snippets:
pulsar.utils.httpurl.
Headers
(headers=None, kind='server', strict=False)[source]¶Utility for managing HTTP headers for both clients and servers.
It has a dictionary like interface with few extra functions to facilitate the insertion of multiple header values. Header fields are case insensitive, therefore doing:
>>> h = Headers()
>>> h['Content-Length'] = '1050'
is equivalent to
>>> h['content-length'] = '1050'
Parameters: |
|
---|
This Headers
container maintains an ordering as suggested by
http://www.w3.org/Protocols/rfc2616/rfc2616.html:
The order in which header fields with differing field names are received is not significant. However, it is “good practice” to send general-header fields first, followed by request-header or response-header fields, and ending with the entity-header fields.
—rfc2616 section 4.2
The strict parameter is rarely used and it forces the omission on non-standard header fields.
update
(iterable)[source]¶Extend the headers with an iterable
.
Parameters: | iterable – a dictionary or an iterable over keys, vaues tuples. |
---|
override
(iterable)[source]¶Extend headers by overriding fields form iterable.
Parameters: | iterable – a dictionary or an iterable over keys, vaues tuples. |
---|
get
(key, default=None)[source]¶Get the field value at key
as comma separated values.
For example:
>>> from pulsar.utils.httpurl import Headers
>>> h = Headers(kind='client')
>>> h.add_header('accept-encoding', 'gzip')
>>> h.add_header('accept-encoding', 'deflate')
>>> h.get('accept-encoding')
results in:
'gzip, deflate'
get_all
(key, default=None)[source]¶Get the values at header key
as a list rather than a
string separated by comma (which is returned by the
get()
method).
For example:
>>> from pulsar.utils.httpurl import Headers
>>> h = Headers(kind='client')
>>> h.add_header('accept-encoding', 'gzip')
>>> h.add_header('accept-encoding', 'deflate')
>>> h.get_all('accept-encoding')
results in:
['gzip', 'deflate']
clear
()[source]¶Same as dict.clear()
, it removes all headers.
getheaders
(key)[source]¶Required by cookielib in python 2.
If the key is not available, it returns an empty list.
add_header
(key, values)[source]¶Add values
to key
header.
If the header is already available, append the value to the list.
Parameters: |
|
---|
pulsar.utils.httpurl.
HttpParser
(kind=2, decompress=False, method=None)[source]¶A python HTTP parser.
Original code from https://github.com/benoitc/http-parser
2011 (c) Benoit Chesneau <benoitc@e-engura.org>
WebSocket Protocol is implemented via the Frame
and
FrameParser
classes.
To obtain a frame parser one should use the frame_parser()
function.
pulsar.utils.websocket.
frame_parser
(version=None, kind=0, extensions=None, protocols=None, pyparser=False)[source]¶Create a new FrameParser
instance.
Parameters: |
|
---|
pulsar.utils.websocket.
FrameParser
(version, kind, ProtocolError, extensions, protocols)[source]¶Decoder and encoder for the websocket protocol.
version
¶Optional protocol version (Default 13).
kind
¶encode
(message, final=True, masking_key=None, opcode=None, rsv1=0, rsv2=0, rsv3=0)[source]¶Encode a message
for writing into the wire.
To produce several frames for a given large message use
multi_encode()
method.
pulsar.utils.websocket.
parse_close
(data)[source]¶Parse the body of a close Frame
.
Returns a tuple (code
, reason
) if successful otherwise
raise ProtocolError
.
Configuration utilities which provides pulsar with configuration parameters which can be parsed from the command line. Parsing is implemented using the python argparser standard library module.
pulsar.utils.config.
Config
(description=None, epilog=None, version=None, apps=None, include=None, exclude=None, settings=None, prefix=None, name=None, log_name=None, **params)[source]¶A dictionary-like container of Setting
parameters for
fine tuning pulsar servers.
It provides easy access to Setting.value
attribute by exposing the Setting.name
as attribute.
Parameters: |
|
---|
settings
¶Dictionary of all Setting
instances available in this
Config
container.
Keys are given by the Setting.name
attribute.
params
¶Dictionary of additional parameters which cannot be parsed on the command line
copy_globals
(cfg)[source]¶Copy global settings from cfg
to this config.
The settings are copied only if they were not already modified.
get
(name, default=None)[source]¶Get the value at name
for this Config
container
The returned value is obtained from:
set
(name, value, default=False)[source]¶Set the Setting
at name
with a new value
.
If default
is True
, the Setting.default
is also set.
parser
()[source]¶Create the argparser for this configuration by adding all
settings via the Setting.add_argument()
method.
Return type: | an instance of ArgumentParser . |
---|
on_start
()[source]¶Invoked by a Application
just before starting.
address
¶An address to bind to, only available if a
bind setting has been added to this
Config
container.
copy
(name=None, prefix=None)[source]¶A copy of this Config
container.
If prefix
is given, it prefixes all non
global settings
with it. Used when multiple applications are loaded.
pulsar.utils.config.
Setting
(name=None, flags=None, action=None, type=None, default=None, nargs=None, desc=None, validator=None, app=None, meta=None, choices=None, const=None)[source]¶Class for creating pulsar settings.
Most parameters can be specified on the command line,
all of them on a config
file.
virtual
= None¶If set to True
the settings won’t be loaded.
It can be only used as base class for other settings.
validator
= None¶A validating function for this setting.
It provided it must be a function accepting one positional argument, the value to validate.
value
= None¶The actual value for this setting.
const
= None¶A constant value required by some action and nargs selections
choices
= None¶Restrict the argument to the choices provided.
type
= None¶The type to which the command-line argument should be converted
is_global
= False¶True
only for
global settings.
app
= None¶Setting for a specific Application
.
default
= None¶The default value for this setting.
flags
= None¶List of options strings, e.g. [-f, --foo]
.
action
= None¶The basic type of action to be taken when this argument is encountered at the command line
meta
= None¶Same usage as metavar
in the python argparse
module. It is
the name for the argument in usage message.
nargs
= None¶The number of command-line arguments that should be consumed
short
= None¶Optional shot description string
desc
= None¶Description string
section
= None¶Setting section, used for creating the settings documentation.
on_start
()[source]¶Called when pulsar server starts.
It can be used to perform custom initialization for this
Setting
.
set
(val, default=False)[source]¶Set val
as the value
for this Setting
.
If default
is True
set also the default
value.
pulsar.utils.internet.
SSLContext
(keyfile=None, certfile=None, cert_reqs=0, ca_certs=None, server_hostname=None, protocol=2)[source]¶A picklable SSLContext class
pulsar.utils.internet.
parse_address
(netloc, default_port=8000)[source]¶Parse an internet address netloc
and return a tuple with
host
and port
.
pulsar.utils.internet.
parse_connection_string
(connection_string, default_port=8000)[source]¶Converts the connection_string
into a three elements tuple
(scheme, host, params)
where scheme
is a string, host
could
be a string or a two elements tuple (for a tcp address) and params
a
dictionary of parameters. The default_port
parameter can be used to
set the port if a port is not available in the connection_string
.
For example:
>>> parse_connection_string('http://127.0.0.1:9080')
('http', ('127.0.0.1', 9080), {})
and this example:
>>> parse_connection_string('redis://127.0.0.1:6379?db=3&password=bla')
('redis', ('127.0.0.1', 6379), {'db': '3', 'password': 'bla'})
pulsar.utils.tools.
checkarity
(func, args, kwargs, discount=0)[source]¶Check if arguments respect a given function arity and return
an error message if the check did not pass,
otherwise it returns None
.
Parameters: |
|
---|
Collection of data structures and function used throughout the library.
pulsar.utils.structures.skiplist.
Skiplist
(data=None, unique=False)[source]¶Sorted collection supporting O(log n) insertion, removal, and lookup by rank.
update
(iterable)¶Extend this skiplist with an iterable over
score
, value
pairs.
rank
(score)[source]¶Return the 0-based index (rank) of score
.
If the score is not available it returns a negative integer which
absolute score is the right most closest index with score less than
score
.
remove_range
(start, end, callback=None)[source]¶Remove a range by rank.
This is equivalent to perform:
del l[start:end]
on a python list. It returns the number of element removed.
remove_range_by_score
(minval, maxval, include_min=True, include_max=True, callback=None)[source]¶Remove a range with scores between minval
and maxval
.
Parameters: |
|
---|---|
Returns: | the number of elements removed. |
pulsar.utils.structures.zset.
Zset
(data=None)[source]¶Ordered-set equivalent of redis zset.
remove
(item)[source]¶Remove item
for the zset
it it exists.
If found it returns the score of the item removed.
Utilities for HTML and text manipulation.
pulsar.utils.html.
HTML_AMPERSAND
= '&'¶HTML & symbol.
pulsar.utils.html.
HTML_EMDASH
= '—'¶HTML – symbol.
pulsar.utils.html.
HTML_ENDASH
= '–'¶HTML - symbol.
pulsar.utils.html.
HTML_GREATER_THEN
= '>'¶HTML > symbol.
pulsar.utils.html.
HTML_LESS_THEN
= '<'¶HTML < symbol.
pulsar.utils.html.
HTML_NON_BREACKING_SPACE
= ' '¶HTML non breaking space symbol.
pulsar.utils.html.
NOTHING
= ('', '', None)¶Tuple of elements considered as null.
pulsar.utils.html.
escape
(html, force=False)[source]¶Returns the given HTML with ampersands, quotes and angle brackets encoded.
A slugify function which handle unicode
Module containing utilities and mixins for logging and serialisation.
pulsar.utils.log.
LocalMixin
[source]¶Defines the local
attribute.
Classes derived from a LocalMixin
can use the
local_method()
and local_property()
decorators for managing
attributes which are not picklable.
local
¶A lazy pulsar.utils.structures.AttributeDictionary
.
This attribute is removed when pickling an instance.
lock
¶A local threading.Lock.
pulsar.utils.log.
WritelnDecorator
(stream)[source]¶Used to decorate file-like objects with a handy ‘writeln’ method. taken from python.
pulsar.utils.log.
configured_logger
(name, config=None, level=None, handlers=None, rootlevel=None, replace=False)[source]¶Configured logger.
pulsar.utils.log.
local_method
(f)[source]¶Decorator to be used in conjunction with LocalMixin
methods.
pulsar.utils.log.
local_property
(f)[source]¶Decorator to be used in conjunction with LocalMixin
methods.
Stand alone compact module for managing python paths.
pulsar.utils.path.
Path
[source]¶A lightweight utility for the filesystem and python modules.
add2python
(module=None, up=0, down=None, front=False, must_exist=True)[source]¶Add a directory to the python path.
Parameters: |
|
---|