Contents¶

ODIN documentation¶

Object Data Mapper for Python

First Steps¶

Quick overview to get up to speed with Odin.

Working with resources¶

Resources are the basic building block of Odin.

Resources: Field reference | Resource Meta options
Loading and Saving: JSON Codec | TOML Codec | YAML Codec | CSV Codec | MessagePack Codec
Mapping: Mapping Classes
Adapters: Adapters
Documenting: Sphinx Extension

See the Odin Examples section for examples on how to use features of Odin.

Extras¶

Integration¶

Project Links¶

Indices and tables¶

Getting started¶

Quick overview to get up to speed with Odin.

Creating resources¶

Odin was designed to solve several common development problems:

Loading data from file or data stream and converting into an object graph
Validating data to ensure it meets the parameters of the program
Mapping/transforming data into a different format or structure
Do all of the above in a way that is easy to maintain, read and test

The goal of this document is to give you an overview of how to use the tools provided by Odin to accomplish the goals set out in the previous paragraph.

Design your resources¶

The resource syntax offers many rich ways of representing your resources. Here’s a quick example:

import odin

class Author(odin.Resource):
    name = odin.StringField()

class Book(odin.Resource):
    title = odin.StringField()
    author = odin.DictAs(Author)
    genre = odin.StringField(max_length=255)
    num_pages = odin.IntegerField(min_value=1)

From this we can see that a resource is a collection of fields, each field maps to a specific data type and accepts options that describe validation rules and how data is handled.

The above example also demonstrates relationships between the two resources. This simple example allows for an other object to be attached to a book. With these simple primitives complex data-structures can be built.

Working with resources¶

With a resource defined the API can be used to work with resources:

# Import the resources we created from our "library" app
>>> from library.resources import Author, Book

# Create an instance of an Author
>>> a = Author(name="Iain M. Banks")

# Create an instance of a Book
>>> b = Book(title="Consider Phlebas", author=a, genre="Space Opera", num_pages=471)
>>> b
<Book: library.resources.Book resource>

# Fields are represented as attributes on the Python object.
>>> b.title
'Consider Phlebas'

# DictAs fields are references to other resources.
>>> b.author.name
'Iain M. Banks'

# Get all the data as a dict
>>> a.to_dict()
{'name': 'Iain M. Banks'}

# Validate that the information entered is valid.
# Create an instance of a Book
>>> b = Book(title="Consider Phlebas", genre="Space Opera", num_pages=471)
>>> b.full_clean()
ValidationError: {'author': [{'name': ['This field cannot be null.']}]}

Loading and saving data¶

Saving and loaded of resources is handled by codecs for each format.

JSON data¶

JSON data is loaded and saved using the odin.codecs.json_codec module. This module exposes an API that is very similar to Pythons built in json module.

Using the Book and Author resources presented in the Creating resources section:

# Import the resources we created from our "library" app
>>> from library.resources import Author, Book

# Create an instance of an Author
>>> a = Author(name="Iain M. Banks")

# Create an instance of a Book
>>> b = Book(title="Consider Phlebas", author=a, genre="Space Opera", num_pages=471)

# Dump as a JSON encoded string
from odin.codecs import json_codec
>>> json_codec.dumps(b)
'{"genre": "Space Opera", "title": "Consider Phlebas", "num_pages": 471, "$": "library.resources.Book", "author": {
"name": "Iain M. Banks", "$": "library.resources.Author"}}'

Note

Note the $ entries. The $ symbol is used to keep track of an objects type in the serialised state and to aid deserialization of data.

Similarly data can be deserialized back into an object graph using the odin.codecs.json_codec.loads() method.

Mapping between resources¶

Defining a mapping¶

Given the following resources:

import odin

class CalendarEvent(odin.Resource):
    name = odin.StringField()
    start_date = odin.DateTimeField()

class CalendarEventFrom(odin.Resource):
    name = odin.StringField()
    event_date = odin.DateTimeField()
    event_hour = odin.IntegerField()
    event_minute = odin.IntegerField()

A mapping can be defined to map from a basic Event to the EventFrom:

class CalendarEventToEventFrom(odin.Mapping):
    from_resource = CalendarEvent
    to_resource = CalendarEventFrom

    # Simple mappings (From Field, Transformation, To Field)
    mappings = (
        odin.define(from_field='start_date', to_field='event_date'),
    )

    # Mapping to multiple fields
    @odin.map_field(to_field=('event_hour', 'event_minute'))
    def start_date(self, v):
        # Return a tuple that is mapped to fields defined as to_fields
        return v.hour, v.minute

When a field name is matched on both resources it will be automatically mapped, for other fields mappings need to be specified along with a transformation method or alternatively a method with a map_field() decorator can be used to handle more complex mappings.

Hint

Both simple mappings and map_field() can accept multiple fields as input and output, although care must be taken to ensure that the transformation method accepts and returns the same number of parameters as is defined in the mapping.

Converting between resources¶

Once a mapping has been defined the Resource.convert_to() or Mapping.apply() are used to convert between object, in addition Mapping.update() can be used to update a existing object:

# Create and instance of a CalendarEvent
>>> event = CalendarEvent(
    name='Launch Party',
    start_date=datetime.datetime(2014, 01, 11, 22, 30))

# Convert to CalendarEventFrom
>>> event_from = event.convert_to(CalendarEventFrom)
>>> event_from
<CalendarEventFrom: example.resources.CalendarEventFrom resource>

>>> event.to_dict()
{'event_date': datetime.datetime(2014, 01, 11, 22, 30),
 'event_hour': 22,
 'event_minute': 30,
 'name': 'Launch Party'}

# Or use the mapping definition CalendarEventToEventFrom
>>> event_from = CalendarEventToEventFrom.apply(event)

# Or update an an existing resource
event.name = 'Grand Launch Party'
event.update_existing(event_from)

>>> event.to_dict()
{'event_date': datetime.datetime(2014, 01, 11, 22, 30),
 'event_hour': 22,
 'event_minute': 30,
 'name': 'Grand Launch Party'}

# Similarly the mapping definition can also be used
>>> CalendarEventToEventFrom(event).update(event_from)

Polymorphism and abstract resources¶

Todo : To be written

API Reference¶

Codecs¶

To load and save data from external data files Odin provides a number of codecs to facilitate the process.

Often the API used by the codec mirrors or extends the API provided by the library the Odin extends on. For example for JSON data Odin provides the same load, loads style interface.

CSV Codec¶

Codec for serialising and de-serialising sequences of resources into CSV format.

The CSV CODEC differs from many other CODECS in that data can be streamed, or read using an iterator.

CSV Codec¶

Codec for iterating a CSV file and parsing into a Resource.

The CSV codec is codec that yields multiple resources rather than a single document. The CSV codec does not support nesting of resources.

Reading data from a CSV file:

with open("my_file.csv") as f:
    with resource in csv_codec.reader(f, MyResource):
        ...

class odin.codecs.csv_codec.Reader(f, resource_type, full_clean=True, error_callback=None, **reader_kwargs)[source]¶

Customisable reader object.

csv_dialect = 'excel'¶: CSV Dialect to use; defaults to the CSV libraries default value of excel.

csv_reader()¶: CSV Reader object to use (if you wish to use unicodecsv or similar)

default_empty_value = ''¶: The default value to use if a field is empty. This can be used to default to None.

extra_field_names¶: Extra fields not included in header

field_mapping¶: Index mapping of CSV fields to resource fields.

field_names¶: Field names from resource.

ignore_header_case = False¶: Use case-less comparison on header fields.

includes_header = True¶: File is expected to include a header.

strict_fields = False¶: Strictly check header fields.

odin.codecs.csv_codec.dump(f, resources, resource_type=None, include_header=True, cls=<built-in function writer>, **kwargs)[source]¶

Dump resources into a CSV file.

Parameters:	f – File to dump to. resources – Collection of resources to dump. resource_type – Resource type to use for CSV columns; if None the first resource will be used. include_header – Write a CSV header. cls – Writer to use when writing CSV, this should be based on `csv.writer`. kwargs – Additional parameters to be supplied to the writer instance.

odin.codecs.csv_codec.dump_to_writer(writer, resources, resource_type=None, fields=None)[source]¶

Dump resources to a CSV writer interface.

The interface should expose the csv.writer interface.

Parameters:	writer (`csv.writer`) – Writer object fields – List of fields to write resources – Collection of resources to dump. resource_type – Resource type to use for CSV columns; if None the first resource will be used.
Returns:	List of fields that where written to.

odin.codecs.csv_codec.dumps(resources, resource_type=None, cls=<built-in function writer>, **kwargs)[source]¶

Dump output to a string

Parameters:	resources – resources – Collection of resources to dump. resource_type – Resource type to use for CSV columns; if None the first resource will be used. cls – Writer to use when writing CSV, this should be based on `csv.writer`. kwargs – Additional parameters to be supplied to the writer instance.

odin.codecs.csv_codec.reader(f, resource, includes_header=False, csv_module=<module 'csv' from '/home/docs/.pyenv/versions/3.7.9/lib/python3.7/csv.py'>, full_clean=True, ignore_header_case=False, strict_fields=False, **kwargs)[source]¶

CSV reader that returns resource objects

Parameters:	f – file like object resource – includes_header – File includes a header that should be used to map columns csv_module – Specify an alternate csv module (eg unicodecsv); defaults to the builtin csv as this module is implemented in C. full_clean – Perform a full clean on each object ignore_header_case – Ignore the letter case on header strict_fields – Extra fields cannot be provided.
Returns:	Iterable reader object
Return type:	Reader

odin.codecs.csv_codec.value_fields(resource)[source]¶: Iterator to get non-composite (eg value) fields for export

JSON Codec¶

Codec for serialising and de-serialising JSON data. Supports both array and objects for mapping into resources or collections of resources.

The JSON codec uses the simplejson module if available and falls back to the json module included in the Python standard library.

Methods¶

odin.codecs.json_codec.load(fp, resource=None, full_clean=True, default_to_not_supplied=False)[source]¶

Load a from a JSON encoded file.

See loads() for more details of the loading operation.

Parameters:

fp – a file pointer to read JSON data from.

resource – A resource type, resource name or list of resources and names to use as the base for creating a resource. If a list is supplied the first item will be used if a resource type is not supplied.

full_clean – Do a full clean of the object as part of the loading process.

default_to_not_supplied – Used for loading partial resources. Any fields not supplied are replaced with NOT_SUPPLIED.

Returns:
A resource object or object graph of resources loaded from file.

odin.codecs.json_codec.loads(s, resource=None, full_clean=True, default_to_not_supplied=False)[source]¶

Load from a JSON encoded string.

If a resource value is supplied it is used as the base resource for the supplied JSON. I one is not supplied a resource type field $ is used to obtain the type represented by the dictionary. A ValidationError will be raised if either of these values are supplied and not compatible. It is valid for a type to be supplied in the file to be a child object from within the inheritance tree.

Parameters:

s – String to load and parse.

resource – A resource type, resource name or list of resources and names to use as the base for creating a resource. If a list is supplied the first item will be used if a resource type is not supplied.

full_clean – Do a full clean of the object as part of the loading process.

default_to_not_supplied – Used for loading partial resources. Any fields not supplied are replaced with NOT_SUPPLIED.

Returns:
A resource object or object graph of resources parsed from supplied string.

odin.codecs.json_codec.dump(resource, fp, cls=<class 'odin.codecs.json_codec.OdinEncoder'>, **kwargs)[source]¶

Dump to a JSON encoded file.

Parameters:

resource – The root resource to dump to a JSON encoded file.

cls – Encoder to use serializing to a string; default is the OdinEncoder.

fp – The file pointer that represents the output file.

odin.codecs.json_codec.dumps(resource, cls=<class 'odin.codecs.json_codec.OdinEncoder'>, **kwargs)[source]¶

Dump to a JSON encoded string.

Parameters:

resource – The root resource to dump to a JSON encoded file.

cls – Encoder to use serializing to a string; default is the OdinEncoder.

Returns:
JSON encoded string.

Customising Encoding¶

Serialisation of Odin resources is handled by a customised json.Encoder. Additional data types can be appended to the odin.codecs.json_codec.JSON_TYPES dictionary.

Example usage¶

Loading a resource from a file:

from odin.codecs import json_codec

with open('my_resource.json') as f:
    resource = json_codec.load(f)

MessagePack Codec¶

Codec for serialising and de-serialising data in MessagePack format. Supports both array and objects for mapping into resources or collections of resources.

The MessagePack codec uses the msgpack-python module.

Methods¶

Customising Encoding¶

Serialisation of Odin resources is handled by a customised msgpack.Packer. Additional data types can be appended to the odin.codecs.msgpack_codec.TYPE_SERIALIZERS dictionary.

Example usage¶

Loading a resource from a file:

from odin.codecs import msgpack_codec

with open('my_resource.msgp') as f:
    resource = msgpack_codec.load(f)

TOML Codec¶

Codec for serialising and de-serialising TOML data. Supports both array and objects for mapping into resources or collections of resources.

The TOML codec uses the toml module.

Methods¶

Customising Encoding¶

Serialisation of Odin resources is handled by a customised toml.Encoder. Additional data types can be appended to the odin.codecs.toml_codec.TOML_TYPES dictionary.

Example usage¶

Loading a resource from a file:

from odin.codecs import toml_codec

with open('my_resource.toml') as f:
    resource = toml_codec.load(f)

YAML Codec¶

Codec for serialising and de-serialising YAML data. Supports both array and objects for mapping into resources or collections of resources.

The YAML codec uses the yaml module and uses the compiled C versions of the library if available.

Methods¶

Customising Encoding¶

Serialisation of Odin resources is handled by a customised yaml.Dumper. Additional data types can be appended to the odin.codecs.yaml_codec.YAML_TYPES dictionary.

Example usage¶

Loading a resource from a file:

from odin.codecs import yaml_codec

with open('my_resource.yaml') as f:
    resource = yaml_codec.load(f)

XML Codec¶

Codec for serialising and de-serialising XML data. Supports both array and objects for mapping into resources or collections of resources.

XML Codec (output only)

Output a resource structure as XML

XML has a unique attribute in the form of the text. This is plain text that can be placed within a pair of tags.

To support this the XML codec includes a TextField, any TextField found on a resource will be exported (if multiple are defined they will all be exported into the one text block).

The TextField is for all intents and purposes just a StringField, other codecs will export any value as a String.

Methods¶

odin.codecs.xml_codec.dump(fp, resource, line_ending='')[source]¶

Dump a resource to a file like object. :param fp: File pointer or file like object. :param resource: Resource to dump :param line_ending: End of line character to apply

odin.codecs.xml_codec.dumps(resource, **kwargs)[source]¶

Dump a resource to a string.

Parameters: resource – Resource to dump

Unsupported Fields¶

There is no direct representation for a odin.fields.DictField.

Example usage¶

Loading a resource from a file:

from odin.codecs import xml_codec

with open('my_resource.xml') as f:
    resource = xml_codec.load(f)

Saving a resource to a file:

from odin.codecs import xml_codec

with open('my_resource.xml', 'w') as f:
    xml_codec.dump(f, resource)

contrib packages¶

Odin provides integration with other Python libraries to simplify common development situations. These packages are kept separate from the rest of Odin to only require dependencies on other libraries if you so chose to use this additional functionality.

This code lives in odin/contrib in the Odin distribution.

Generating resource documentation¶

Note

When using document generation Jinja2 is required.

Odin has built in support for generating documentation of resources that have been registered. This is where the various verbose_name, doc_text and doc strings are used generate documentation.

Quick example¶

The default documentation format is reStructuredText, to enable easy integration with Sphinx for producing project documentation.

A basic example:

from odin import doc_gen
import my_project.resources  # Import required resources so they get registered

with file("resources.rst", "w") as fp:
    doc_ren.dump(fp)

The resources.rst file can now be registered into your Sphinx documentation.

Doc-gen API¶

The documentation generation API consists of two methods in the vain of the main Odin API:

odin.doc_gen.dump - Output documentation to a file, requires a file pointer.
odin.doc_gen.dumps - Return the documentation as a string

Both methods take the same optional parameters.

fmt

Format of the output, by default this is RESTRUCTURED_TEXT.

There are not currently any other options available.

exclude

List of resources to exclude when generating documentation.

template_path

Template path to include in template search path, this is used to customise the look of outputted templates.

See Customising output for more information.

Customising output¶

Geographic Values¶

Fields and data types that handle latitude and longitude values.

todo:	This section is in progress

Datatypes¶

latitude¶

A latitude value. A latitude is a value between -90.0 and 90.0.

longitude¶

A longitude value. A longitude is a value between -180.0 and 180.0.

latlng¶

Combination latitude and longitude value.

point¶

A point in cartesian space. This type can be either 2D (on a plain) or 3D (includes a z-axis).

Fields¶

LatitudeField¶

class LatitudeField([min_value=None, max_value=None, **options])

A latitude.

LatitudeField has two extra arguments:

LatitudeField.min_value: The minimum latitude that can be accepted (within the range of a latitude).
LatitudeField.max_value: The maximum latitude that can be accepted (within the range of a latitude).

LongitudeField¶

class LongitudeField([min_value=None, max_value=None, **options])

A longitude.

LongitudeField has two extra arguments:

LongitudeField.min_value: The minimum longitude that can be accepted (within the range of a longitude).
LongitudeField.max_value: The maximum longitude that can be accepted (within the range of a longitude).

LatLngField¶

class LatLngField([**options])

A latlng.

PointField¶

class PointField([**options])

A point.

Currency and Money Values¶

Fields and data types that handle money values.

todo:	This section is in progress

Datatypes¶

Amount¶

Combines an value and a Currency to represent a monetary amount.

Currency¶

Defines a currency and maintains metadata about the currency.

Fields¶

AmountField¶

class AmountField([allowed_currencies=None, min_value=None, max_value=None, **options])

An amount.

AmountField has three extra arguments:

AmountField.allowed_currencies: The currencies that can be accepted by this field, value is enforced Odin’s validation. If None is supplied any currency is acceptable.
AmountField.min_value: The minimum amount that can be accepted.
AmountField.max_value: The maximum amount that can be accepted.

Physical Quantities with Pint¶

Additional resources fields that include a unit.

Note

This contrib module depends on the Pint units library this can be installed with:

pip install pint

Fields¶

FloatField¶

class FloatField([max_length=None, **options])

A float.

FloatField has one extra argument:

Sphinx Extension¶

Odin provides a Sphinx extension for documenting resources. It behaves similarly to the builtin Autodoc tools.

Setup¶

Add the Odin extension into your Sphinx conf.py:

extensions = [
    ...
    'odin.contrib.sphinx',
    ...
]

Usage¶

Extension to sphinx.ext.autodoc to support documenting Odin resources.

This extension supports output in two forms, class doc form (for documenting the python objects) and API form for documenting information about fields for use in an API document.

Usage:

.. autoodin-resource:: namespace.path.to.your.Resource
    :include_virtual:
    :hide_choices:

To select API form use the include_virtual option.

Options¶

The following options are provided by the documenter:

include_virtual virtual fields should be included in docs
include_validators validators should be listed for fields
hide_choices don’t include the list of valid choices for fields

Exceptions¶

Exceptions defined by Odin.

exception odin.exceptions.CodecDecodeError[source]¶: Exception raised by a codec during a decoding operation.

exception odin.exceptions.CodecEncodeError[source]¶: Exception raised by a codec during an encoding operation.

exception odin.exceptions.CodecError[source]¶: Exception raised by a codec during an operation.

exception odin.exceptions.InvalidPathError(path, *args, **kwargs)[source]¶: Raised when a path is invalid (eg referencing an unknown field)

exception odin.exceptions.MappingError[source]¶: Exceptions related to mapping, will typically be a more specific MappingSetupError or MappingExecutionError.

exception odin.exceptions.MappingExecutionError[source]¶: Exception raised during the execution of mapping rules.

exception odin.exceptions.MappingSetupError[source]¶: Exception raised during the setup of mapping rules.

exception odin.exceptions.MultipleMatchesError(path, *args, **kwargs)[source]¶: When traversing a path to get a single value, a filtering operation matched multiple values.

exception odin.exceptions.NoMatchError(path, *args, **kwargs)[source]¶: When traversing a path to get a value no match was found.

exception odin.exceptions.ResourceDefError[source]¶: Exceptions raised if a resource definition contains errors.

exception odin.exceptions.ResourceException(message, code=None, params=None)[source]¶

Errors raised when generating resource from files.

Exception inherits from ValidationError for backwards compatibility.

exception odin.exceptions.TraversalError(path, *args, **kwargs)[source]¶: Exception raised during a traversal operation.

exception odin.exceptions.ValidationError(message, code=None, params=None)[source]¶: An error while validating data.

Mapping¶

Resource Mapping API reference. For introductory material, see Mapping between resources.

Mapping Classes¶

A mapping is a utility class that defines how data is mapped between objects, these objects are typically odin.Resource but other Python objects can be supported.

The basics:

Each mapping is a Python class that subclasses odin.Mapping.
Each mapping defines a from_obj and a to_obj
Parameters and decorated methods define rules for mapping between objects.

Defining a Mapping¶

`from_obj` and `to_obj`¶

These attributes specify the source and destination of the mapping operation and must be defined to for a mapping to be considered valid.

Note

For an object to be successfully mapped it’s fields need to be known. This is handled by a FieldResolver instance.

Example:

class AuthorToNewAuthor(odin.Mapping):
    from_obj = Author
    to_obj = NewAuthor

Auto generated mappings¶

When a field of the same name exists on both the from and to objects an automatic mapping is created. If no data transformation or data type conversion is needed no further work is required. The mappings will be automatically created.

You can exclude fields from automatic generation by; including the field in the exclude_fields list() (or tuple()). Fields are also excluded from automatic generation when they are specified as a target by any other mapping rule.

Fields that defined relationships (eg DictAs and ArrayOf) are also handled by the auto generated mapping process, provided that a mapping has been defined previously for the Resource types that these relations refer to. There is one special case however, this is where Resource types match; in this situation the odin.mapping.helpers.NoOpMapper is used to transparently copy the items.

Applying transformations to data¶

Complex data manipulation can be achieved by passing a field value through a mapping function.

The simplest way to apply a transformation action is to use the map_field() decorator.

Example:

class AuthorToNewAuthor(odin.Mapping):
    from_obj = Author
    to_obj = NewAuthor

    @odin.map_field
    def title(self, value):
        return value.title()

In this simple example we are specifying a mapping of the title field that ensures that the title value is title case.

But what about if we want to split a field into multiple outputs or combine multiple values into a single field? Not a problem, transformation actions can accept and return multiple values. These values just need to specified:

class AuthorToNewAuthor(odin.Mapping):
    from_obj = Author
    to_obj = NewAuthor

    @odin.map_field(from_field='name', to_field=('first_name', 'last_name'))
    def split_author_name(self, value):
        first_name, last_name = value.split(' ', 1)
        return first_name, last_name

Conversely we could combine these fields:

class AuthorToNewAuthor(odin.Mapping):
    from_obj = Author
    to_obj = NewAuthor

    @odin.map_field(from_field=('first_name', 'last_name'))
    def name(self, first_name, last_name):
        return "%s %s" % (first_name, last_name)

While this example is extremely simplistic it does demonstrate the flexibility of mapping rules. Not also that a value for the to_field is not specified, the mapping decorators will default to using the method name as the to or from field if it is not specified.

Odin includes several decorators that preform handle different mapping scenarios.

map_field¶

Decorate a mapping class method to mark it as a mapping rule.

from_field: The string name or a tuple of names of the field(s) to map from. The function that is being decorated must accept the same number of parameters as fields specified.
to_field: The string name or tuple of names of the field(s) to map to. The function that is being decorated must return a tuple with the same number of parameters as fields specified.

map_list_field¶

Decorate a mapping class method to mark it as a mapping rule. This decorator works in much the same way as the basic map_field except rather than treating the response as a set of fields it treats it as a list result. This allows you to map list of objects.

from_field: The string name or a tuple of names of the field(s) to map from. The function that is being decorated must accept the same number of parameters as fields specified.
to_field: The string name of tuple of names of the field(s) to map to. The function that is being decorated must return a tuple with the same number of parameters as fields specified.

assign_field¶

This is a special decorator that allows you to generate a value that is assigned to the resulting field.

to_field: The string name or tuple of names of the field(s) to map to. The function that is being decorated must return a tuple with the same number of parameters as fields specified.

Low level mapping¶

The final way to specify a mapping is by generating the actual mapping rules directly. A basic mapping rule is a three part tuple that contains the name of the from field (or tuple of multiple source fields) a transform action or None if there is no transform required and finally the name of the two field (or tuple of multiple destination fields).

Note

The number of input parameters and number of parameters returned by the action methods must much the number defined in the mapping. A odin.exceptions.MappingExecutionError will be raised if an incorrect number of parameters is specified.

A list of mapping rules:

class AuthorToNewAuthor(odin.Mapping):
    from_obj = Author
    to_obj = NewAuthor

    mappings = (
        ('dob', None, 'date_of_birth'),
    )

Tip

Use the define() and assign() methods to simplify the definition of mappings. They provides many sensible defaults.

While the basic mapping only includes and source, action and destination definitions the mappings structure actually supports three additional boolean parameters. These are to_list, bind and skip_if_none.

to_list¶

The two list option is what the map_list_field decorator uses to indicate the the returned object is a list value.

bind¶

For use with action methods defined outside the mapping class, if bind is set to True the mapping instance is passed to the action method as the first parameter.

skip_if_none¶

This flag changes the way that values that are None are handled. If set to True if the from value is None the value will not be supplied to the destination object allowing the destination objects defaulting process to handle the value.

Mapping Instances¶

Mapping instances provide various methods to aid in the mapping process.

Initialisation¶

The init method accepts a source object and an optional context. The context is a defaults to a dict() and allows any value to be stored or supplied to the mapping.

When using the apply class method to map a list of objects the context is used to track the index count.

`convert`¶

Method that starts the mapping process and returns a populated to_obj.

`update`¶

Update an existing object with fields from the provided by the source_obj.

`diff`¶

Compare the field values from the source_obj with a supplied destination and return all the fields that differ.

`loop_idx`¶

A convenience property gives access to the current loop index when converting a list of objects.

`loop_depth`¶

A convenience property that provides the nested loop depth of the current mapping operation.

`in_loop`¶

A convenience property that indicates if the current mapping operation is in a loop.

Mapping Factories¶

When mapping between two objects that are similar eg between a Django model and a resource, or between versions of resources.

odin.mapping.mapping_factory(from_obj, to_obj, base_mapping=<class 'odin.mapping.Mapping'>, generate_reverse=True, mappings=None, reverse_mappings=None, exclude_fields=None, reverse_exclude_fields=None, register_mappings=True)[source]¶

Factory method for generating simple mappings between objects.

A common use-case for this method is in generating mappings in baldr’s model_resource_factory method that auto-generates resources from Django models.

Parameters:

from_obj – Object to map from.

to_obj – Object to map to.

base_mapping – Base mapping class; default is odin.Mapping.

generate_reverse – Generate the reverse of the mapping ie swap from_obj and to_obj.

mappings – User provided mappings (this is equivalent ot odin.Mapping.mappings)

reverse_mappings – User provided reverse mappings (this is equivalent ot odin.Mapping.mappings). Only used if generate_reverse is True.

exclude_fields – Fields to exclude from auto-generated mappings

reverse_exclude_fields – Fields to exclude from auto-generated reverse mappings.

register_mappings – Register mapping in mapping lookup cache.

Returns:
if generate_reverse is True a tuple(forward_mapping, reverse_mapping); else just the forward_mapping.

Return type:
Mapping | (Mapping, Mapping)

There is also the simpler method when only a forward mapping is desired.

odin.mapping.forward_mapping_factory(from_obj, to_obj, base_mapping=<class 'odin.mapping.Mapping'>, mappings=None, exclude_fields=None)[source]¶

Factory method for generating simple forward mappings between objects.

Parameters:

from_obj – Object to map from.

to_obj – Object to map to.

base_mapping – Base mapping class; default is odin.Mapping.

mappings – User provided mappings (this is equivalent ot odin.Mapping.mappings)

exclude_fields – Fields to exclude from auto-generated mappings

Returns:
Forward_mapping object.

Mapping Helpers¶

Odin includes a few helpers to help defining mappings.

Defining Mappings¶

When defining mappings using the shorthand mappings property these methods simplify the definition of mapping rules. They also provide sensible defaults.

odin.mapping.define(from_field=None, action=None, to_field=None, to_list=False, bind=False, skip_if_none=False)[source]¶

Helper method for defining a mapping.

Parameters:

from_field – Source field to map from.
action – Action callable to perform during mapping, accepted fields differ based on options.
to_field – Destination field to map to; if not specified the from_field
to_list – Assume the result is a list (rather than a multi value tuple).
bind – During the mapping operation the first parameter should be the mapping instance.
skip_if_none – If the from field is None do not include the field (this allows the destination object to define it’s own defaults etc)

Returns:

A mapping definition.

odin.mapping.assign(to_field, action, to_list=False, bind=True, skip_if_none=False)[source]¶

Helper method for defining an assignment mapping.

Parameters:

to_field – Destination field to map to; if not specified the from_field
action – Action callable to perform during mapping, accepted fields differ based on options.
to_list – Assume the result is a list (rather than a multi value tuple).
bind – During the mapping operation the first parameter should be the mapping instance; defaults to True
skip_if_none – If the from field is None do not include the field (this allows the destination object to define it’s own defaults etc)

Returns:

A mapping definition.

Resources¶

Resource API reference. For introductory material, see Creating Resources.

Field reference¶

This section contains all the details of the resource fields built into Odin.

Field options¶

The following arguments are available to all field types. All are optional.

verbose_name¶

Field.verbose_name

A human-readable name for the field. If the verbose name isn’t given, Odin will automatically create it using the field’s attribute name, converting underscores to spaces.

verbose_name_plural¶

Field.verbose_name_plural

A human-readable name for the field. If the verbose name isn’t given, Odin will automatically create it using the field’s attribute name, converting underscores to spaces.

name¶

Field.name

Name of the field as it appears in the exported document. If the name isn’t given, Odin will use the field’s attribute name.

null¶

Field.null

If True Odin will raise a validation error if a value is null. Default is False.

choices¶

Field.choices

An iterable (e.g., a list or tuple) of 2-tuples to use as choices for this field. If this is given, the choices are used to validate entries.

Note

Choices are also used by the odin.contrib.doc_gen to generate documentation.

A choices list looks like this:

GENRE_CHOICES = (
    ('sci-fi', 'Science Fiction'),
    ('fantasy', 'Fantasy'),
    ('others', 'Others'),
)

The first element in each tuple is the value that will be used to validate, the second element is used for documentation. For example:

import Odin

class Book(Odin.Resource):
    GENRE_CHOICES = (
        ('sci-fi', 'Science Fiction'),
        ('fantasy', 'Fantasy'),
        ('others', 'Others'),
    )
    title = Odin.StringField()
    genre = Odin.StringField(choices=GENRE_CHOICES)
>>> b = Book(title="Consider Phlebas", genre="sci-fi")
>>> b.genre
'sci-fi'

default¶

Field.default

The default value for the field. This can be a value or a callable object. If callable it will be called every time a new object is created.

doc_text (help_text)¶

Field.doc_text

Doc text is used by the odin.contrib.doc_gen to generate documentation.

Note

help_text will be deprecated in a future release in favor of doc_text.

Also useful for inline documentation even if documentation is not generated.

validators¶

Field.validators

error_messages¶

Field.error_messages

is_attribute¶

Field.is_attribute

use_default_if_not_provided¶

Field.use_default_if_not_provided

Standard fields¶

Simple field types.

StringField¶

class StringField([max_length=None, **options])

A string.

StringField has one extra argument:

StringField.max_length: The maximum length (in characters) of the field. The max_length value is enforced Odin’s validation.

IntegerField¶

class IntegerField([min_value=None, max_value=None, **options])

An integer.

IntegerField has two extra arguments:

IntegerField.min_value: The minimum value of the field. The min_value value is enforced Odin’s validation.
IntegerField.max_value: The maximum value of the field. The max_value value is enforced Odin’s validation.

FloatField¶

class FloatField([**options])

A floating-point number represented in Python by a float instance.

FloatField has two extra arguments:

FloatField.min_value: The minimum value of the field. The min_value value is enforced Odin’s validation.
FloatField.max_value: The maximum value of the field. The max_value value is enforced Odin’s validation.

BooleanField¶

class BooleanField([**options])

A true/false field.

DateField¶

class DateField([**options])

A date field or date encoded in ISO-8601 date string format.

TimeField¶

class TimeField([assume_local=True, **options])

A datetime.time field or time encoded in ISO-8601 time string format.

TimeField has an extra argument:

TimeField.assume_local: This adjusts the behaviour of how a naive time (time objects with no timezone) or time strings with no timezone specified. By default assume_local is True, in this state naive time objects are assumed to be in the current system timezone. Similarly on decoding a time string the output time will be converted to the current system timezone.

NaiveTimeField¶

class NaiveTimeField([ignore_timezone=False, **options])

A datetime.time field or time encoded in ISO-8601 time string format. The naive time field differs from TimeField in the handling of the timezone, a timezone will not be applied if one is not specified.

NaiveTimeField has an extra argument:

NaiveTimeField.ignore_timezone: Ignore any timezone information provided to the field during parsing. Will also actively strip out any timezone information when processing an existing time value.

DateTimeField¶

class DateTimeField([**options])

A datetime.datetime field or date encoded in ISO-8601 datetime string format.

DateTimeField has an extra argument:

DateTimeField.assume_local: This adjusts the behaviour of how a naive time (date time objects with no timezone) or date time strings with no timezone specified. By default assume_local is True, in this state naive datetime objects are assumed to be in the current system timezone. Similarly on decoding a date time string the output datetime will be converted to the current system timezone.

NaiveDateTimeField¶

class NaiveDateTimeField([ignore_timezone=False, **options])

A datetime.datetime field or time encoded in ISO-8601 datetime string format. The naive date time field differs from DateTimeField in the handling of the timezone, a timezone will not be applied if one is not specified.

NaiveDateTimeField has an extra argument:

NaiveDateTimeField.ignore_timezone: Ignore any timezone information provided to the field during parsing. Will also actively strip out any timezone information when processing an existing datetime value.

HttpDateTimeField¶

class HttpDateTimeField([**options])

A datetime field or date encoded in ISO-1123 or HTTP datetime string format.

UUIDField¶

class UUIDField([**options])

A UUID field.

This field supports most accepted values for initializing a UUID except bytes_le.

EnumField¶

class EnumField(enum, [**options])

Note

This field requires Python >=3.4 or the enum34 package.

A enum.Enum field that will convert to and from an enum and it’s native type.

Ensure that the enum value is compatible with the codec being used.

The enum.IntEnum variant is also supported.

Changed in version 1.5.0: Choices can be used with EnumField to specify a subset of options. A sequence of enum values should be used that will be converted to choice tuples by Odin.

ArrayField¶

class ArrayField([**options])

An array structure represented in Python by a list instance.

TypedArrayField¶

class TypedArrayField(field, [**options])

An array structure represented in Python by a list instance accepts an additional parameter of another field type that each entry in the array is validated against.

TypedArrayField.field: An instance of an odin field that is used to validate each entry in the array.

TypedDictField¶

class TypedDictField(key_field, value_field, [**options])

A object structure represented in Python by a dict instance accepts additional parameters of both a key and value field type that each item in the dict is validated against.

TypedDictField.key_field: An instance of an odin field that is used to validate each key in the dict; default is StringField.
TypedDictField.value_field: An instance of an odin field that is used to validate each value in the dict; default is StringField.

DictField¶

class DictField([**options])

A dictionary.

Note

The object values in the object are not defined.

Composite fields¶

Odin also defines a set of fields that allow for composition.

DictAs field¶

class DictAs(of[, **options])

A child object. Requires a positional argument: the class that represents the child resource.

Note

A default dict is automatically assigned.

ArrayOf field¶

class ArrayOf(of[, **options])

A child list. Requires a positional argument: the class that represents a list of resources.

Note

A default list is automatically assigned.

DictOf field¶

class DictOf(of[, **options])

A child dict. Requires a positional argument: the class that represents a dict (or hash map) of resources.

Note

A default dict is automatically assigned.

Virtual fields¶

Virtual fields are special fields that can be used to calculate a value or provide a value lookup. Unlike using a property a virtual field is also a treating like field in that it can be mapped or exported.

Note

You can use the

Virtual fields share many of the options of regular fields:

verbose_name
verbose_name_plural
name
doc_text (help_text)

Calculated field¶

class CalculatedField(expr[, **options])

Resources Instances¶

A resource is a collection of fields and associated meta data that is used to validate data within the fields.

The basics:

Each resource is a Python class that subclasses odin.Resource.
Each field attribute of the resource is defined by a odin.Field subclass.

Quick example¶

This example resource defines a Book, which has a title, genre and num_pages:

import odin

class Book(odin.Resource):
    title = odin.StringField()
    genre = odin.StringField()
    num_pages = odin.IntegerField()

title, genre and num_pages are fields. Each field is specified as a class attribute.

The above Book resource would create a JSON object like this:

{
    "$": "resources.Book",
    "title": "Consider Phlebas",
    "genre": "Space Opera",
    "num_pages": 471
}

Some technical notes:

The $ field is a special field that defines the type of Resource.
The name of the resource, resources.Book, is automatically derived from some resource metadata but can be overridden.

Fields¶

The most important part of a resource – and the only required part of a resource – is the list of fields it defines. Fields are specified by class attributes. Be careful not to choose field names that conflict with the resources API like clean.

Example:

class Author(odin.Resource):
    name = odin.StringField()

class Book(odin.Resource):
    title = odin.StringField()
    authors = odin.ArrayOf(Author)
    genre = odin.StringField()
    num_pages = odin.IntegerField()

Field types¶

Each field in your resource should be an instance of the appropriate Field class. ODIN uses the field class types to determine a few things:

The data type (e.g. Integer, String).
Validation requirements.

Field options¶

Each field takes a certain set of field-specific arguments (documented in the resource field reference).

There’s also a set of common arguments available to all field types. All are optional. They’re fully explained in the reference, but here’s a quick summary of the most often-used ones:

null

If True, the field is allowed to be null. Default is False.

default

The default value for the field. This can be a value or a callable object. If callable it will be called every time a new object is created.

choices

An iterable (e.g., a list or tuple) of 2-tuples to use as choices for this field.

A choices list looks like this:

GENRE_CHOICES = (
    ('sci-fi', 'Science Fiction'),
    ('fantasy', 'Fantasy'),
    ('others', 'Others'),
)

The first element in each tuple is the value that will be stored field or serialised document, the second element is a display value.

Again, these are just short descriptions of the most common field options.

Verbose field names¶

Each field type, except for DictAs and ArrayOf, takes an optional first positional argument – a verbose name. If the verbose name isn’t given, Odin will automatically create it using the field’s attribute name, converting underscores to spaces.

In this example, the verbose name is “person’s first name”:

first_name = odin.StringField("person's first name")

In this example, the verbose name is “first name”:

first_name = odin.StringField()

DictAs and ArrayOf require the first argument to be a resource class, so use the verbose_name keyword argument:

publisher = odin.DictAs(Publisher, verbose_name="the publisher")
authors = odin.ArrayOf(Author, verbose_name="list of authors")

Resource level validation¶

Field validation can be customised at the resource level. This is useful as it allows validation of data using a value from another field for example ensuring a maximum value is greater than a minimum value, or checking that a password and a check password matches.

This is achieved using by defining a method called clean_FIELDNAME which accepts a single value argument, Odin will then use this method during the cleaning process to validate the field. Odin will then use the value that is returned from the clean method, allowing you to apply any customised formatting. If an issue is found with a value then raise a odin.exceptions.ValidationError and the error returned will be applied to validation results.

Example:

class Timing(odin.Resource):
    minimum_delay = odin.IntegerField(min_value=0)
    maximum_delay = odin.IntegerField()

    def clean_maximum_delay(self, value):
        if value < self.minimum_delay:
            raise ValidationError('Maximum delay must be greater than the minimum delay value')
        return value

Important

Ensure that a return value is provided, if no return value is specified the Python default is None and this is the value that Odin will use.

Relationships¶

To really model more complex documents objects and lists need to be able to be combined, Odin offers ways to define these structures, DictAs and ArrayOf fields handle these structures.

DictAs relationships¶

To define a object-as relationship, use odin.DictAs. You use it just like any other Field type by including it as a class attribute of your resource.

DictAs requires a positional argument: the class to which the resource is related.

For example, if a Book resource has a Publisher – that is, a single Publisher publishes a book:

class Publisher(odin.Resource):
    # ...

class Book(odin.Resource):
    publisher = odin.DictAs(Publisher)
    # ...

This would produce a JSON document of:

{
    "$": "resources.Book",
    "title": "Consider Phlebas",
    "publisher": {
        "$": "resources.Publisher",
        "name": "Macmillan"
    }
}

ArrayOf relationships¶

To define a array-of relationship, use odin.ArrayOf. You use it just like any other Field type by including it as a class attribute of your resource.

ArrayOf requires a positional argument: the class to which the resource is related.

For example, if a Book resource has a several Authors – that is, a multiple authors can publish a book:

class Author(odin.Resource):
    # ...

class Book(odin.Resource):
    authors = odin.ArrayOf(Author)
    # ...

This would produce a JSON document of:

{
    "$": "resources.Book",
    "title": "Consider Phlebas",
    "authors": [
        {
            "$": "resources.Author",
            "name": "Iain M. Banks"
        }
    ]
}

Resource inheritance¶

Resource inheritance in Odin works almost identically to the way normal class inheritance works in Python. The only decision you have to make is whether you want the parent resources to be resources in their own right, or if the parents are just holders of common information that will only be visible through the child resources.

Abstract base classes¶

Abstract base classes are useful when you want to put some common information into a number of other resources. You write your base class and put abstract=True in the Meta class. This resource will then not be able to created from a JSON document. Instead, when it is used as a base class for other resources, its fields will be added to those of the child class.

An example:

class CommonBook(odin.Resources):
    title = odin.StringField()

    class Meta:
        abstract = True

class PictureBook(CommonBook):
    photographer = odin.StringField()

The PictureBook resource will have two fields: title and photographer. The CommonBook resource cannot be used as a normal resource, since it is an abstract base class.

todo:	Add details of how to support multiple object types in a list using Abstract resources

Resource Meta options¶

Meta options are flags that can be applied to a resource to affect it’s behaviour or how it is dealt with via Odin tools.

Give your resource metadata by using an inner class Meta, eg:

class Book(odin.Resource):
    class Meta:
        name_space = "library"
        verbose_name_plural = "Books"

    title = odin.StringField()

Resource metadata is “anything that’s not a field”, module_name and human-readable plural names (verbose_name and verbose_name_plural). None are required, and adding class Meta to a resource is completely optional.

Meta Options¶

name

Override the name of a resource. This is the codecs when serialising/de-serialising as a name to represent the resource. The default name is the name of the class used to define the resource.

name_space

The name space is an optional string value that is used to group a set of common resources. Typically a namespace should be in the form of dot-atoms eg: university.library or org.poweredbypenguins. The default is no namespace.

verbose_name

A long version of the name for used when displaying a resource or in generated documentation. The default verbose_name is a name attribute that has been converted to lower case and spaces put before each upper case character eg: LibraryBook -> “library book”

verbose_name_plural

A pluralised version of the verbose_name. The default is to use the verbose name and append an ‘s’ character. In the case of many words this does not work correctly so this attribute allows for the default behaviour to be overridden.

abstract

Marks the current resource as an abstract resource. See the section Abstract base classes for more detail of the abstract attribute. The default value for abstract is False.

doc_group

A grouping for documentation purposes. This is purely optional but is useful for grouping common elements together. The default value for doc_group is None.

type_field

The field used to identify the object type during serialisation/de-serialisation. This defaults to the $ character.

key_field_name

Used by external libraries like baldr for identifying what field is used as the key field to uniquely identify a resource instance (a good example would be an ID field).

key_field_names

Similar to the key_field_name but for defining multi-part keys.

field_sorting

Used to customise how fields are sorted (primarily affects the order fields will be exported during serialisation) during inheritance. The default behaviour is to sort fields in the child resource before appending the fields from the parent resource(s).

Settings this option to True will cause field sorting to happen after all of the fields have been attached using the default sort method. The default method sorts the fields by the order they are defined.

Supplying a callable allows for customisation of the field sorting eg sort by name:

def sort_by_name(fields):
    return sorted(fields, key=lambda f: f.name)

class MyResource(Resource):
    class Meta:
        field_sorting = sort_by_name

user_data

Additional data that can be added to metadata. This can be used to provide additional parameters beyond those supported by odin for a custom application use-case.

For example:

class MyResource(Resource):
    class Meta:
        user_data = {
            "custom": "my-custom-value",
        }

Adapters¶

Adapters are wrappers around resources that allow for an alternate view of the data contained in a resource or provide additional functionality that is specific to part of your code base. An adapter can be used just like the plain resource with most methods within Odin, including codecs.

See Adapter Examples

Validators¶

Built in validators¶

These validators are provided with odin.

class odin.validators.RegexValidator(regex=None, message=None, code=None)[source]¶

class odin.validators.URLValidator(regex=None, message=None, code=None)[source]¶

class odin.validators.MaxValueValidator(limit_value)[source]¶

class odin.validators.MinValueValidator(limit_value)[source]¶

class odin.validators.LengthValidator(limit_value)[source]¶

class odin.validators.MaxLengthValidator(limit_value)[source]¶

class odin.validators.MinLengthValidator(limit_value)[source]¶

Odin also includes a helper method for generating simple validators.

odin.validators.simple_validator(assertion=None, message='The supplied value is invalid', code='invalid')[source]¶

Create a simple validator.

Parameters:	assertion – An Validation exception will be raised if this check returns a none True value. message – Message to raised in Validation exception if validation fails. code – Code to included in Validation exception. This can be used to customise the message at the resource level.

Usage:

>>> none_validator = simple_validator(lambda x: x is not None, message="This value cannot be none")

This can also be used as a decorator:

@simple_validator(message="This value cannot be none")
def none_validator(v):
    return v is not None

Traversal¶

Traversal package provides tools for iterating and navigating a resource tree.

TraversalPath¶

Todo: In progress…

ResourceTraversalIterator¶

Iterator for traversing (walking) a resource structure, including traversing composite fields to fully navigate a resource tree.

This class has hooks that can be used by subclasses to customise the behaviour of the class:

on_enter - Called after entering a new resource.

on_exit - Called after exiting a resource.

Integration¶

Integrations of Odin with other libraries/services.

Integration with Amazon Web Services¶

Integration with Amazon Web Services is via odincontrib.aws <https://github.com/python-odin/odincontrib.aws/

Requires boto3.

Odin Contrib AWS includes:

Integration with DynamoDB. Dynamo versions of fields, resources (Table) and querying

Dynamo DB¶

Tables can be defined using extended Resources, this provides an interface for table creation, save (put) items, query, scan and batch operations. Querying is performed either using raw parameters (as used by Boto3) or using a SQLAlchemy style fluent interface.

Example:

from odincontrib_aws import dynamodb


class Book(dynamodb.Table):
    class Meta:
        namespace = 'library'

    title = dynamodb.StringField()
    isbn = dynamodb.StringField(key=True)
    num_pages = dynamodb.IntegerField()
    genre = dynamodb.StringField(choices=(
        ('sci-fi', 'Science Fiction'),
        ('fantasy', 'Fantasy'),
        ('biography', 'Biography'),
        ('others', 'Others'),
        ('computers-and-tech', 'Computers & technology'),
    ))

session = dynamodb.Session()

# Save a new book into a Dynamo DB table
book = Book(
    title="The Hitchhiker's Guide to the Galaxy",
    isbn="0-345-39180-2",
    num_pages=224,
    genre='sci-fi',
)
session.put_item(book)

# Scan through all books in the table (this method is transparently paged)
for book in session.scan(Book):
    print(book.title)

SQS¶

This is development in progress to use Odin for defining and verifying SQS messages.

Integration with Django¶

Integration with Django is via baldr.

Baldr includes:

ResourceField for saving a resource to a Django model.
Field resolver for Django models, this allows you to map between Django models and Odin resources.
RESTful API implementation using Odin resources.

ResourceField¶

This is a field that handles serialisation/deserialisation of Odin resources from a database. Data is serialised as JSON. It is basically a Django django.db.models.fields.TextField with the additional required option resource.

Example:

class MyModel(models.Model):
    my_resource = ResourceField(MyResource)

Field resolver¶

By including baldr as an application in Django the field resolver is automatically registered. From there you can write mappings between Django models and Odin resources. Set either the to_resource or from_resource fields to be Django models and that’s it.

RESTful API¶

One of the powerful features of Odin is validation of data that is loaded. This makes Odin perfect for handling RESTful API’s.

todo:	Expand on capabilities.

Odin Examples¶

Examples of odin usage to get you started.

Adapter Examples¶

Filtering out fields¶

There are many times where certain fields need to be filtered from a resource before the resource is serialised (either to file or into an HTTP response). This could be to remove any sensitive information, or internal debug fields that should not be returned to an API. The odin.adapters.ResourceAdapter class makes this trivial.

Which fields should be filtered can be defined in one of two ways:

# At instantiation to allowing for dynamic filtering
my_filtered_resource = odin.ResourceAdapter(my_resource, exclude=('password',))

# Of predefined if this adapter will be used multiple times (not an adapter is not tied to any particular resource
# type so a single adapter could be used to filter the password field from any resource)
class PasswordFilter(odin.ResourceAdapter):
    exclude = ['password',]

my_filtered_resource = PasswordFilter(my_resource)

Added additional functionality to a resource¶

I have a resource that define a set of simulation results. I want to be able to render the results of this simulation into several different formats eg an HTML table, a text file, a chart.

One approach would be to add to a method to the resource for each of the formats I wich to render like to_html or to_text, however this has a number of drawbacks:

The resource is not more complex with an additional method for each of the target formats
Rendering code is now mixed in with your data structure definition
In a larger team there is more chances of multiple people working on the same file making merges more complex
If I define another resource for a different set of results, sharing common code is harder
The resource has a different interface for each method

The odin.adapters.ResourceAdapter class simplifies this.

By defining a rendering adapter for each of the different targets, the rendering code for each of these targets is encapsulated in a single class:

class HtmlRenderingAdapter(odin.ResourceAdapter):
    def render(self):
        ...


class TextRenderingAdapter(odin.ResourceAdapter):
    def render(self):
        ...

The benefits of this approach:

All features required for each render adapter are encapsulated
Each of these adapters can exist in their own file
Both adapters include the same render interface they can be passed to code that understands that interface
Both classes can both inherit off a common base class that provides common code that is related to each rendering operation.

CSV Codec Examples¶

The CSV codec is different to other codecs in that it produces an iterable of odin.Resource objects rather than a single structure.

Typical Usage¶

When using the odin.codecs.csv_codec my recommendation is to define your CSV file format using a CSV csv.Dialect and a sub-class of the odin.codecs.csv_codec.Reader to configure how files are handled.

A simple example of a customised csv.Dialect and odin.codecs.csv_codec.Reader:

import csv

from odin.codecs import csv_codec


class MyFileDialect(csv.Dialect):
    """
    Custom CSV dialect
    """
    delimiter = '\t'       # Tab delimited
    quotechar = '"'        # Double quotes for quoting
    lineterminator = '\n'  # UNIX style line termination


class MyFileReader(csv_codec.Reader):
    """
    Custom file reader
    """
    # Specify our custom dialect
    csv_dialect = MyFileDialect
    # Header case is not important
    ignore_header_case = True
    # Treat empty values as `None`
    default_empty_value = None

This can then be used with:

# To read a file
with open("my_file.csv") as f:
    for r in MyFileReader(f, MyResource):
        pass

# To write a file
with open("my_file.csv", "w") as f:
    csv_codec.dump(f, my_resources, dialect=MyFileDialect)

Handling errors¶

As resources are generated as each CSV row is read any validation errors raised also need to be handled. The default behavior is for a validation error to be raised when a invalid row is encountered effectively stopping processing. However, it may be valid to skip bad rows or, a report of bad rows can be generated and processing continued.

There are two approaches that can be used:

Provide an error_callback value to the odin.codecs.csv_codec.Reader
Sub-class the odin.codecs.csv_codec.Reader class and implement a

custom handle_validation_error method to process errors.

Note

Providing an error_callback will overwrite any custom handle_validation_error method.

The error_callback option is the simplest:

def error_callback(validation_error, idx):
    """
    Handle errors reading rows from CSV file.

    :param validation_error: The validation error exception
    :param idx: The row index the error occurred on.
    :returns: ``None`` or ``False`` to explicitly case the exception to
        be raised.

    """
    print("Error in row {}: {}".format(idx, validation_error), file=sys.stderr)

with open("my_file.csv") as f:
    for r in MyFileReader(f, MyResource, error_callback=error_callback):
        ...

The sub-class method is more involved upfront but does allow for more customisation:

class MyReader(csv_codec.Reader):
    """
    Custom file reader that reports errors to a file.
    """
    def __init__(self, f, error_file, *args, **kwargs):
        super().__init__(self, *args, **kwargs)

        self.error_file = error_file

    def handle_validation_error(self, validation_error, idx):
        self.error_file.write("{}\t{}\n".format(idx, validation_error))


with open("my_file.csv") as f_in, open("my_file.error.csv", "w") as f_err:
    for r in MyReader(f_in, f_err, MyResource):
        ...

The second option allows for a lot of customisation and reuses. For example the error report could itself output a CSV file.

Contents¶

ODIN documentation¶

First Steps¶

Working with resources¶

Extras¶

Integration¶

Project Links¶

Indices and tables¶

Getting started¶

Creating resources¶

Design your resources¶

Working with resources¶

Loading and saving data¶

JSON data¶

Mapping between resources¶

Defining a mapping¶

Converting between resources¶

Polymorphism and abstract resources¶

API Reference¶

Codecs¶

CSV Codec¶

CSV Codec¶

JSON Codec¶

Methods¶

Customising Encoding¶

Example usage¶

MessagePack Codec¶

Methods¶

Customising Encoding¶

Example usage¶

TOML Codec¶

Methods¶

Customising Encoding¶

Example usage¶

YAML Codec¶

Methods¶

Customising Encoding¶

Example usage¶

XML Codec¶

Methods¶

Unsupported Fields¶

Example usage¶

contrib packages¶

Generating resource documentation¶

Quick example¶

Doc-gen API¶

Customising output¶

Geographic Values¶

Datatypes¶

latitude¶

longitude¶

latlng¶

point¶

Fields¶

LatitudeField¶

LongitudeField¶

LatLngField¶

PointField¶

Currency and Money Values¶

Datatypes¶

Amount¶

Currency¶

Fields¶

AmountField¶

Physical Quantities with Pint¶

Fields¶

FloatField¶

Sphinx Extension¶

Setup¶

Usage¶

Options¶

Exceptions¶

Mapping¶

Mapping Classes¶

Defining a Mapping¶

from_obj and to_obj¶

Auto generated mappings¶

Applying transformations to data¶

Low level mapping¶

Mapping Instances¶

`from_obj` and `to_obj`¶

`convert`¶

`update`¶

`diff`¶

`loop_idx`¶

`loop_depth`¶

`in_loop`¶