Documentation for python-stdnet's DEVELOPMENT version. Get the release docs here.

Internals and Utilities

Default Settings


Stdnet comes with a bunch of extendible utilities for serializing model data into different formats.

Get serializer

stdnet.odm.get_serializer(name, **options)

Retrieve a serializer register as name. If the serializer is not available a ValueError exception will raise. A common usage pattern:

qs = MyModel.objects.query().sort_by('id')
s = odm.get_serializer('json')

Register serializer

stdnet.odm.register_serializer(name, serializer)

Register a new serializer to the library.

  • name – serializer name (it can override existing serializers).
  • serializer – an instance or a derived class of a stdnet.odm.Serializer class or a callable.


class stdnet.odm.Serializer(**options)

The stdnet serializer base class. During initialization, the options dictionary is used to override the default_options. These are specific to each Serializer implementation.


Dictionary of default options which are overwritten during initialisation. By default it is an empty dictionary.


Dictionary of options.


CList of data to dump into a stream.


Add a Query qs into the collection of data to dump into a stream. No writing is done until the write() method.


Write the serialized data into a stream. If stream is not provided, a python StringIO is used.

Returns:the stream object.
load(models, stream, model=None)

Load a stream of data into the database.

  • models – the Mapper which must contains all the model this method will load.
  • stream – bytes or an object with a read method returning bytes.
  • model – Optional StdModel we need to load. If not provided all models in stream are loaded.

This method must be implemented by subclasses.


class stdnet.odm.JsonSerializer(**options)

The default Serializer of stdnet. It serialise/unserialise models into json data. It has one option given by the indent of the json string for pretty serialisation.

on_load_model(model, model_data)

Callback when a model is about to be loaded. If it returns the model, the model will get loaded otherwise it will skip the loading.


Callback when loading of data is finished

JSON utilities


class stdnet.utils.jsontools.JSONDateDecimalEncoder(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

The default JSON encoder used by stdnet. It provides JSON serialization for four additional classes:

  • as a {'__date__': timestamp} dictionary
  • datetime.datetime as a {'__datetime__': timestamp} dictionary
  • decimal.Decimal as a {'__decimal__': number} dictionary

See also

It is the default encoder for stdnet.odm.JSONField



The default JSON decoder hook. It is the inverse of stdnet.utils.jsontools.JSONDateDecimalEncoder.


stdnet.utils.jsontools.flat_to_nested(data, instance=None, attname=None, separator=None, loads=None)

Convert a flat representation of a dictionary to a nested representation. Fields in the flat representation are separated by the splitter parameters.

  • data – a flat dictionary of key value pairs.
  • instance – optional instance of a model.
  • attribute – optional attribute of a model.
  • separator – optional separator. Default "__".
  • loads – optional data unserializer.
Return type:

a nested dictionary



Multiply dictionaries by a numeric values and add them together.

  • series

    a tuple of two elements tuples. Each serie is of the form:


    where weight is a number and dictionary is a dictionary with numeric values.

  • skip – optional list of field names to skip.

Only common fields are aggregated. If a field has a non-numeric value it is not included either.


Classes used for encoding and decoding stdnet.odm.Field values.

class stdnet.utils.encoders.Encoder

Virtaul class for encoding data in data structures. It exposes two methods for encoding and decoding data to and from the data server.


The type of data once loaded into python


Serialize data for database


Unserialize data from database


True if this Encoder requires a stdnet.odm.Session.

load_iterable(iterable, session=None)

Load an iterable.

By default it returns a generator of data loaded via the loads() method.


an iterable over decoded data.

These are all available Encoder:

class stdnet.utils.encoders.NoEncoder

A dummy encoder class

class stdnet.utils.encoders.Default(charset='utf-8', encoding_errors='strict')

The default unicode encoder. It converts bytes to unicode when loading data from the server. Viceversa when sending data.

class stdnet.utils.encoders.NumericDefault(charset='utf-8', encoding_errors='strict')

It decodes values into unicode unless they are numeric, in which case they are decoded as such.

class stdnet.utils.encoders.Bytes(charset='utf-8', encoding_errors='strict')

The binary encoder

class stdnet.utils.encoders.Json(charset='utf-8', encoding_errors='strict', json_encoder=None, object_hook=None)

A JSON encoder for maintaning python types when dealing with remote data structures.

class stdnet.utils.encoders.PythonPickle(protocol=2)

A safe pickle serializer. By default we use protocol 2 for compatibility between python 2 and python 3.

class stdnet.utils.encoders.DateTimeConverter

Convert to and from python datetime objects and unix timestamps

class stdnet.utils.encoders.DateConverter


class stdnet.StdNetException

A general StdNet exception

class stdnet.ImproperlyConfigured

‘A stdnet.StdNetException raised when stdnet is somehow improperly configured

class stdnet.QuerySetError

A stdnet.StdNetException raised by a stdnet.odm.Query.

class stdnet.FieldError

Generic Field error

class stdnet.FieldValueError

A stdnet.FieldError raised when passing a wrong value to a field. This exception is cought during the model instance validation algorithm in stdnet.odm.base.Metaclass.is_valid().


Stdnet includes a signal dispatcher which helps allow decoupled applications get notified when actions occur elsewhere in the framework. In a nutshell, signals allow certain senders to notify a set of receivers that some action has taken place. They are especially useful when many pieces of code may be interested in the same events.

The data mapper provide with the following built-in signals in the stdnet.odm module:

  • pre_commit triggered before new instances or changes on existing instances are committed to the backend server.
  • post_commit triggered after new instances or changes on existing instances are committed to the backend server.

It is also possible to add callback to single instances in the following way:

instance = MyModel(...)



stdnet.utils.populate(datatype='string', size=10, start=None, end=None, converter=None, choice_from=None, **kwargs)

Utility function for populating lists with random data. Useful for populating database with data for fuzzy testing. Supported data-types

  • string

    For example:

    populate('string',100, min_len=3, max_len=10)

    create a 100 elements list with random strings with random length between 3 and 10

  • date

    For example:

    from datetime import date
    populate('date',200, start = date(1997,1,1), end =

    create a 200 elements list with random objects between start and end

  • integer

    For example:

    populate('integer',200, start = 0, end = 1000)

    create a 200 elements list with random int between start and end

  • float

    For example:

    populate('float', 200, start = 0, end = 10)

    create a 200 elements list with random floats between start and end

  • choice (elements of an iterable)

    For example:

    populate('choice', 200, choice_from = ['pippo','pluto','blob'])

    create a 200 elements list with random elements from choice_from.


Test case classes and plugins for stdnet testing. Requires pulsar.


class stdnet.utils.test.TestCase(methodName='runTest')

A unittest.TestCase subclass for testing stdnet with synchronous and asynchronous connections. It contains several class methods for testing in a parallel test suite.


class attribute which indicates which backend can run the test. There are several options:

  • multipledb = False The test case does not require a backend and only one TestCase class is added to the test-suite regardless of which backend has been tested.
  • multipledb = True, the default falue. Create as many TestCase classes as the number of backend tested, each backend will run the tests.
  • multipledb = string, list, tuple, Only those backend will run tests.

A stdnet.BackendDataServer for this TestCase class. It is a class attribute which is different for each TestCase class and it is created by the setUpClass() method.


A DataGenerator class for creating data. The data is created during the setUpClass() class method.


The DataGenerator instance created from data_cls.


The default StdModel for this test. A class attribute.


A tuple of models which can be registered by this test. The model is always the model at index 0 in models.


A Mapper with all models registered with backend.


alias of DataGenerator

classmethod backend_params()

Optional backend parameters for tests.

classmethod setUpClass()

Set up this TestCase before test methods are run. here is where a backend server instance is created and it is unique for this TestCase class. It create the mapper, a Mapper with all models registered. There shouldn’t be any reason to override this method, use after_setup() class method instead.

classmethod after_setup()

This class method can be used to setup this TestCase class after the setUpClass() was called. By default it does nothing.

classmethod session(**kwargs)

Create a new stdnet.odm.Session bind to the TestCase.backend attribute.

classmethod query(model=None)

Shortcut function to create a query for a model.

classmethod multi_async(iterable, **kwargs)

Treat iterable as a container of asynchronous results.

assertEqualId(instance, value, exact=False)

Assert the value of a primary key in a backend agnostic way.

  • instance – the StdModel to check the primary key value.
  • value – the value of the id to check against.
  • exact – if True the exact value must be matched. For redis backend this parameter is not used.


class stdnet.utils.test.DataGenerator(size, sizes=None)

A generator of data. It must be initialised with the size parameter obtained from the command line which is avaiable as a class attribute in TestCase.


A dictionary of sizes for this generator. It is a class attribute with the following entries: tiny, small, normal, big and huge.


The actual size of the data to be generated. Obtained from the sizes and the input size code during initialisation.


Called during initialisation to generate the data. kwargs are additional key-valued parameter passed during initialisation. Must be implemented by subclasses.

populate(datatype='string', size=None, **kwargs)

A shortcut for the stdnet.utils.populate() function. If size is not given, the size is used.

random_string(min_len=5, max_len=30)

Return a random string