References¶

Complete, technical information for the significant classes, methods and functions in the library, appears here. There is also full coverage of the type expressions that are passed to File and Folder objects, at creation time.

Type Expressions¶

The simplest example of a type expression is when the name of a registered class is passed as an argument:

f = ar.File('job', ReceivedJob)

There is a desirable clarity in such code - the file named job contains a representation of a ReceivedJob. For several reasons including ease-of-use and polymorphism, the argument passed to File is not limited to just names of registered classes. It is actually a facillity for describing the content of the job file in a more general and extensible way. Those descriptions are known as type expressions.

A type expression is an instance of one of the following:

a basic Python type, e.g. float
one of the basic Ansar types, e.g. WorldTime
one of the Ansar container types, e.g. MapOf
a registered class (refer to A Quick Tour and More About Types)
an object reference (refer to Really Complicated Documents)
an internal Ansar type.

The container types and the object reference type require parameters that include further instances of type expressions. This is a recursive definition that allows for expression of arbitrarily complex types.

Basic Python Types¶

The Python types that are accepted in type expressions, along with their respective Ansar equivalents, are listed below. Internally all Python types are converted to their Ansar equivalents before use.

Either set of types can be used. Using the Python types is probably more Pythonic.

Python	Ansar	Notes
`bool`	`Boolean`	True or false.
`int`	`Integer8`	A large, signed integer.
`float`	`Float8`	A large, floating-point number.
`bytearray`	`Block`	A sequence of arbitrary bytes.
`bytes`	`String`	A sequence of printable bytes.
`str`	`Unicode`	A sequence of Unicode code-points.
`uuid.UUID`	`UUID`	A UUID from the standard Python library.

Refer to the following section for more information, especially relating to the significance of the different “string” types.

The Ansar Types¶

Full set of Ansar types appears below.

Ansar	Python	Notes
Boolean	`bool`	True or false.
Byte	`int`	A single arbitrary byte.
Character	`bytes`	A single printable byte.
Rune	`str`	A single Unicode code-point.
Integer	`int`	Integer2/Integer4/Integer8. Signed integer numbers consuming 2-8 bytes.
Unsigned	`int`	Unsigned2/Unsigned4/Unsigned8. Unsigned, integer numbers consuming 2-8 bytes.
Float	`float`	Float4/Float8. Signed, floating- point numbers consuming 4 or 8 bytes.
Enumeration	`int`	Names for operational numbers.
Block	`bytearray`	A sequence of arbitrary bytes.
String	`bytes`	A sequence of printable bytes.
Unicode	`str`	A sequence of Unicode code-points.
WorldTime	`float`	A Python time, i.e. `float`.
ClockTime	`float`	A Python time, i.e. `float`.
TimeSpan	`float`	A time delta, i.e. t2 - t1.
UUID	`UUID`	A Python uuid.UUID.
ArrayOf(type, size)	`list`	Fixed number of objects.
VectorOf(type)	`list`	A sequence of zero or more objects.
SetOf(type)	`set`	A collection of unique objects.
MapOf(key, type)	`dict`	A collection of key-value pairs.
DequeOf(type)	`deque`	A double-ended queue of objects.
UserDefined(type)		An instance of a registered class.
Any		An instance of any registered class.
PointerTo(type)		An object that may appear multiple times in the single representation.
Type	`class`	A registered class.
Word		A generic form.
Address		Internal use.
TargetAddress		Internal use.

Note

The Ansar type system is a portable type system that must consider other programming languages and machine architectures. Further information on this topic is outside the scope of this document. Suffice to say that the library accepts the Basic Python Types precisely so that Python developers can work within the Python type system, as much as possible.

Strings Of Things¶

The Byte, Character and Rune types facilitate the proper handling of an arbitrary byte, a printable byte and a Unicode code-point, respectively. There are no exact Python equivalents for these types as Python stores these values as “strings of length 1”. They can be used in type expressions for finer control over the representation of those short strings.

The Block, String and Unicode types describe sequences of Byte, Character and Rune, respectively.

The String and Block types result in different representations of the same application data, i.e. a sequence of bytes. The former assumes that there is a benefit to passing on the printable bytes (0x20 through to 0x7E) without alteration, i.e. for readability. The non-printing bytes will be “escaped” using the mechanism appropriate to the current encoding.

The Block type is intended for the handling of binary data, such as the block-by-block transfer of image files. Blocks of bytes are always base64-encoded within representations.

Warning

Application “string” variables containing values in the range of 0 to 255 are entirely valid. However, the specification for the XML encoding does not provide for the representation of all the values in that range. This affects use of the Python bytes type and the library Character and String types.

Variables that contain exclusively “printable” values such as those values from decimal 32 (space) through to decimal 126 (tilde) are safe when using the XML encoding. Where variables may contain values outside that range, e.g. from decimal zero (nul) to decimal 31 (unit separator) and from decimal 127 (del) through to decimal 255, the behaviour of the XML encoding is not fully defined. There are differences relating to the versions of XML (the library generates exclusively XML 1.0 encodings) and the XML parser in use. Note that the library uses the standard Python XML parser for input, but that other tools in a development environment will include their own parser components.

The bytes, Character and String types are best reserved for “printable” content. If there is explicit need for non-printable content then applications should use the JSON encoding.

Numbers And CPUs¶

The Integer, Unsigned and Float types facillitate the proper handling of the implied number types. The full names for these types include a suffix, e.g. Unsigned4, which refers to the number of bytes consumed by that type. The Python type system manages numbers in a different manner, i.e. without the same regard to portability as Ansar.

All type expressions involving numbers should use the Basic Python Types. The wider set of types offered by Ansar relate to the design of application data that must be portable across multiple programming languages and/or CPU architectures. By using the Basic Python Types, an application ensures that data is portable across Python implementations.

Time Values¶

The WorldTime and ClockTime types provide for readable representations of standard Python time values (i.e. float).

A WorldTime is a GMT or Zulu representation. This is the “safest” representation in that it exhibits “pass-through” behaviour. The floating point value does not change even as the value is stored and recovered many times, potentially by different applications executing in different timezones. A WorldTime conforms to ISO 8601 and looks like this:

2008-07-01T19:01:37Z

A ClockTime uses the same representation, omitting the timezone specification (i.e. Z).

2010-06-06T06:46:01.866233

Recovering a ClockTime produces a value appropriate to the local timezone. A ClockTime implements the concept of a specific time on a particular calendar day - that being a different float value for every timezone in the world.

A TimeSpan is a relative value or delta, the difference between two time values. A few example representations appear below:

1h2m3s
8h
10m
0.0125s
1d

These are readable text representations of float values. After recovery the resulting value can be added to any Python time value and that will have the expected effect, i.e. adding a recovered TimeSpan of 2m will add 120.0 to the Python time value (noting that true time intervals are not nearly as simple as two time values and a bit of arithmetic):

>>> t = ar.clock_epoch('2012-03-06T19:00:30')
>>> t
1331013630.0
>>> d = ar.text_to_span('2m')
>>> d
120.0
>>> t += d
>>> ar.clock_iso(t)
'2012-03-06T19:02:30'

Functions such as clock_epoch, text_to_span and clock_iso are the functions used by the library to perform the translations needed, during store and recover operations.

User Defined Objects¶

The proper type expression for a registered class looks like:

ar.UserDefined(ReceivedJob)

The shorthand form ReceivedJob is accepted. The expression is used to form the proper name-value pairs during storage and perform the proper member assignments during recovery.

The registration of Python classes combined with the UserDefined type expression provides the basis for recovery of fully-realized application types.

To express the concept of “any registered class”, use the following:

ar.Any

Refer to Polymorphic Persistence, in More About Types.

Object Pointers¶

The proper type expression for an object that may appear at multiple points in a single store operation, looks like:

ar.PointerTo(ReceivedJob)

The library uses these “pointers” to manage a collection of shared objects, and to reconstruct references to those objects during a recover operation.

Internal Ansar Types¶

There are a small number of type expressions that exist for internal use. They have limited utility for an application. The Type and Word expressions are used in the implementation of both the Any feature and the Incognito class, the latter used to manage the recovery of unregistered classes.

The Type expression could be used in the following manner:

>>> f = ar.File('registered-class', ar.Type)
>>> f.store(ReceivedJob)
>>>
>>> r, _ = f.recover()
>>> r
<class '__main__.ReceivedJob'>

Note

The parameter passed to the store() method is a Python class not an instance of that class.

Notes On Usage And Implementation¶

The library needs type expressions to be precise, meticulous and complete. Sustaining those standards at all times is difficult, so the library takes an active role by resolving a few common mis-codings.

Using Python Types is Okay¶

Internally the library implements an abstract type system that has broader requirements than only meeting the needs of a Python application. This can be confusing and where there are no broader intentions - e.g. portability of stored materials across languages and CPUs - any such confusion is counter-productive. For this reason the library supports the use of Python types within type expressions. It immediately translates instances of Python types into their library equivalents. There is no runtime difference to the host application.

Type Expressions Are Values¶

There is a fairly subtle difference between what can seem to be a valid type expression and what a properly formed type expression actually needs to be. For this reason all type expressions passed to the library are put through a “clean-it-up” process that makes small adjustments. These are mostly to do with missing parentheses. Consider the type expression:

te = ar.MapOf(ar.Byte, ar.ArrayOf(ar.WorldTime, 4))

This expression is more correctly written as:

te = ar.MapOf(ar.Byte(), ar.ArrayOf(ar.WorldTime(), 4))

The MapOf and ArrayOf objects are expecting further objects as arguments, not the Byte and WorldTime classes that are being passed in the first version. These kinds of errors are insidious and fixable at that point where the expression is presented.

Class Names Are Not Values¶

It is convenient and results in code clarity, but a registered class is not actually a type expression that can be used during operation of the library. Internally all references to registered classes are upgraded to instances of UserDefined. This code:

f = ar.File('job', ReceivedJob)

Becomes this:

f = ar.File('job', ar.UserDefined(ReceivedJob))

The library makes significant use of dispatching during its internal operations, based on an item of application data and its associated type expression. Dispatching is much more efficient if it is dealing with the single possibility of UserDefined rather than the list of all the classes that have been registered within the application.

Classes And Methods¶

`File`¶

class ansar.file.File(name, te, encoding=None, create_default=False, pretty_format=True, decorate_names=True)¶

Store and recover application values using files.

Parameters

name (str) – the name of the file
te (type expression) – formal description of the content
encoding (class) – selection of representation, defaults to CodecJson
create_default (bool) – return default instance on file not found, defaults to False
pretty_format (bool) – generate human-readable file contents, defaults to True
decorate_names (bool) – auto-append an encoding-dependent extension to the file name, defaults to True

store(value)¶

Generate a representation of value and write to the saved name.

Parameters: value – any application value matching the saved te
Returns: none

recover()¶

Read from the saved name, parse and marshal into an application value.

Returns: A 2-tuple of an application value and a version.
Return type: a value matching the saved te and a str

Exceptions raised by `File`¶

class ansar.file.FileFailure(what, name, note, code)¶

Base exception for all file exceptions.

Parameters

what (str) – the failed operation
name (str) – the name of the file
note (str) – a short, helpful description
code (int) – the associated low-level, integer, error code

class ansar.file.FileNotFound(what, name, note, code=0)¶: The named file did not exist.

class ansar.file.FileNoAccess(what, name, note, code=0)¶: No access or a permissions problem.

class ansar.file.FileNotAFile(what, name, note, code=0)¶: Refers to a folder or the path includes a non-folder.

class ansar.file.FileIoFailed(what, name, note, code=0)¶: A device I/O operation failed.

class ansar.file.FileEncoding(what, name, note)¶: File or object content problem, encoding or decoding failed.

Refer also to Codec exceptions

`Folder`¶

class ansar.folder.Folder(path=None, te=None, re=None, encoding=None, pretty_format=True, decorate_names=True, create_default=False, keys_names=None, make_absolute=True)¶

Create and manage a collection of application values, using a folder.

Parameters

path (str) – the location in the filesystem
te (type expression) – formal description of the content
re (a Python regular expression) – formal description of expected file names
encoding (class) – selection of representation, defaults to CodecJson
pretty_format (bool) – generate human-readable file contents, defaults to True
decorate_names (bool) – auto-append an encoding-dependent extension to the file name, defaults to True
keys_names (2-tuple of functions) – a key-composer function and a name-composer function
make_absolute (bool) – expand a relative path to be an absolute location, defaults to True

folder(name, te=None, re=None, encoding=None, pretty_format=None, decorate_names=None, create_default=None, keys_names=None)¶

Create a new Folder object representing a sub-folder at the current location.

Parameters

path (str) – the name to be added to the saved path
te (type expression) – formal description of the content
re (a Python regular expression) – formal description of expected file names
encoding (class) – selection of representation, defaults to CodecJson
pretty_format (bool) – generate human-readable file contents, defaults to True
decorate_names (bool) – auto-append an encoding-dependent extension to the file name, defaults to True
keys_names (2-tuple of functions) – a key-composer function and a name-composer function
make_absolute (bool) – expand a relative path to be an absolute location, defaults to True

Returns

a new location in the filesystem

Return type

Folder

file(name, te, encoding=None, pretty_format=None, decorate_names=None, create_default=None)¶

Create a new File object representing a file at the current location.

Parameters

name (str) – the name to be added to the saved path
te (type expression) – formal description of the content
encoding (class) – selection of representation, defaults to CodecJson
pretty_format (bool) – generate human-readable file contents, defaults to True
decorate_names (bool) – auto-append an encoding-dependent extension to the file name, defaults to True
create_default (bool) – return default instance on file not found, defaults to False

Returns

a new file in the filesystem

Return type

File

matching()¶

A generator method used to scan for files in the folder.

Returns: a sequence of filenames matching the Folder criteria.
Return type: str

each()¶

A generator method used to process the files in the folder.

Returns: a sequence of File objects matching the Folder criteria.
Return type: File

store(values)¶

A method used to store a dict of values as files in the folder.

Parameters: values (dict) – a collection of application values
Returns: None.

recover()¶

A generator method used to recover application values from the files in the folder.

Returns: a sequence of 3-tuples, 0) key, 1) the value and 2) a version tag or None
Return type: a 3-tuple

add(values, item)¶

A method used to add a value, both to a dict of values and as a file in the folder.

Parameters

values (dict) – a collection of application values
item (refer to Folder.te) – the value to be added

update(values, item)¶

A method used to update a value, both in a dict of values and as a file in the folder.

Parameters

values (dict) – a collection of application values
item (refer to Folder.te) – the value to be updated

remove(values, item)¶

A method used to remove a value, both from a dict of values and as a file in the folder.

Parameters

values (dict) – a collection of application values
item (refer to Folder.te) – the value to be removed

clear(values)¶

A method used to remove all values, both from a dict of values and as files in the folder.

Parameters: values (dict) – a collection of application values

erase(name)¶

A method used to delete the named file from the folder.

Parameters: name (str) – a name of a file

exists(name)¶

A method used to detect the named file, within the folder.

Parameters: name (str) – a name of a file
Returns: does the file exist
Return type: bool

key(item)¶

A method used to generate the stable key for a given application value.

Parameters: item – an application value
Returns: the key
Return type: folder dependent

name(item)¶

A method used to generate the stable filename for a given application value.

Parameters: item – an application value
Returns: the filename
Return type: str

Exceptions raised by `Folder`¶

Refer to File exceptions

`Codec`¶

class ansar.json.CodecJson(return_proxy=None, local_termination=None, pretty_format=False, decorate_names=True)¶

Encoding and decoding of JSON representations.

This class is the default value for the encoding parameter, related to the various store and recover operations of the library. Refer to the base codec class for the significant methods.

class ansar.lmx.CodecXml(return_proxy=None, local_termination=None, pretty_format=False, decorate_names=True)¶

Encoding and decoding of XML representations.

This class is an alternative to the JSON codec, in those places where an encoding parameter is passed into the library.

class ansar.codec.Codec(extension, w2t, t2w, return_proxy, local_termination, pretty_format, decorate_names)¶

Base class for all codecs, e.g. CodecJson

Parameters

extension (str) – the additional text added to file names, e.g. json
w2t (function) – the low-level conversion of application data to its text representation
t2w (function) – the low-level parsing of text back to application data.
return_proxy (internal) – an address that the codec will use to transform deserialized addresses.
local_termination (internal) – an address the codec will use to transform deserialized, “to” addresses.
pretty_format (bool) – generate a human-readable layout, defaults to True
decorate_names (bool) – auto-append a dot-extension suffix, defaults to True

encode(value, te)¶

Encode an application value to its stored representation.

Parameters

value – any application value matching the te type expression
te (type expression) – a formal description of the value

Returns

a portable representation

Return type

str

decode(representation, te)¶

Decode a representation to its final application form.

Parameters

representation (str) – the result of a previous encode operation
te (a type expression) – a formal description of memory

Returns

an application value

Exceptions raised by `Codec`¶

class ansar.codec.CodecError(note)¶

Base exception for all codec exceptions.

Parameters: note (str) – a short, helpful description

class ansar.codec.CodecUsage(note, *a)¶

The codec cannot proceed due to its supplied environment such as unusable parameters.

Parameters

note (str) – a short, helpful description
a (list) – values to be substituted into note

class ansar.codec.CodecFailed(note, *a)¶

The codec failed during encoding or decoding activity, such as parsing.

Parameters

note (str) – a short, helpful description
a (list) – values to be substituted into note

`Message And Bind`¶

class ansar.message.Message¶: The base class for all registered application objects.

class ansar.message.Unknown¶: An abstract class used to indicate an unexpected message.

class ansar.message.Incognito(type_name=None, decoded_word=None)¶: A class that holds the recovered materials of an unregistered message.

ansar.message.default_time()¶

A function for initializing a time variable, i.e. WorldTime, ClockTime or TimeSpan.

Returns: a meaningless time value
Return type: float

ansar.message.default_uuid()¶

A function for initializing a UUID variable.

Returns: a global, constant UUID value
Return type: uuid.UUID

ansar.message.default_vector()¶

A function for initializing a vector variable.

Returns: a fresh, empty vector
Return type: list

ansar.message.default_set()¶

A function for initializing a set variable.

Returns: a fresh, empty set
Return type: set

ansar.message.default_map()¶

A function for initializing a map variable.

Returns: a fresh, empty map
Return type: dict

ansar.message.default_deque()¶

A function for initializing a deque variable.

Returns: a fresh, empty double-ended queue
Return type: collections.deque

ansar.message.default_value(te)¶

A function for initializing any variable with an associated type expression.

Parameters: te – see type expression
Returns: an item of application data

ansar.bind.bind(object_type, *args, **kw_args)¶

The function that registers the specified class.

Parameters

object_type (Message-derived class) – the application class to be registered
args (list) – optional positional args
kw_args (dict) – optional key-word args

Exceptions raised by `Message And Bind`¶

class ansar.message.MessageError¶: Base exception for all message exceptions.

class ansar.message.MessageRegistrationError(name, reason)¶

The request to register a class cannot be fulfilled.

Parameters

name (str) – the name of the class
reason (str) – a short description

`Versioning`¶

ansar.version.migrate(f, upgrade, *args, **kw_args)¶

A function for auto-storing objects that change after an upgrade.

Parameters

upgrade (function) – application upgrade code
args (list) – position upgrade args, optional
kw_args (dict) – key-word upgrade args, optional

Returns

the recovered, upgraded and migrated object

References¶

Type Expressions¶

Basic Python Types¶

The Ansar Types¶

Strings Of Things¶

Numbers And CPUs¶

Time Values¶

User Defined Objects¶

Object Pointers¶

Internal Ansar Types¶

Notes On Usage And Implementation¶

Using Python Types is Okay¶

Type Expressions Are Values¶

Class Names Are Not Values¶

Classes And Methods¶

`File`¶

Exceptions raised by `File`¶

`Folder`¶

Exceptions raised by `Folder`¶

`Codec`¶

Exceptions raised by `Codec`¶

`Message And Bind`¶

Exceptions raised by `Message And Bind`¶

`Versioning`¶

Table of Contents

Previous topic

This Page

References¶

Type Expressions¶

Basic Python Types¶

The Ansar Types¶

Strings Of Things¶

Numbers And CPUs¶

Time Values¶

User Defined Objects¶

Object Pointers¶

Internal Ansar Types¶

Notes On Usage And Implementation¶

Using Python Types is Okay¶

Type Expressions Are Values¶

Class Names Are Not Values¶

Classes And Methods¶

File¶

Exceptions raised by File¶

Folder¶

Exceptions raised by Folder¶

Codec¶

Exceptions raised by Codec¶

Message And Bind¶

Exceptions raised by Message And Bind¶

Versioning¶

`File`¶

Exceptions raised by `File`¶

`Folder`¶

Exceptions raised by `Folder`¶

`Codec`¶

Exceptions raised by `Codec`¶

`Message And Bind`¶

Exceptions raised by `Message And Bind`¶

`Versioning`¶