.. _references: References ********** Complete, technical information for the significant classes, methods and functions in the library, appears here. There is also full coverage of the type expressions that are passed to :py:class:`~ansar.file.File` and :py:class:`~ansar.folder.Folder` objects, at creation time. .. _type-expressions: Type Expressions ================ The simplest example of a type expression is when the name of a registered class is passed as an argument: .. code-block:: python f = ar.File('job', ReceivedJob) There is a desirable clarity in such code - the file named ``job`` contains a representation of a ``ReceivedJob``. For several reasons including ease-of-use and polymorphism, the argument passed to :py:class:`~ansar.file.File` is not limited to just names of registered classes. It is actually a facillity for describing the content of the ``job`` file in a more general and extensible way. Those descriptions are known as type expressions. A type expression is an instance of one of the following: * a basic Python type, e.g. ``float`` * one of the basic Ansar types, e.g. ``WorldTime`` * one of the Ansar container types, e.g. ``MapOf`` * a registered class (refer to :ref:`a-quick-tour` and :ref:`more-about-types`) * an object reference (refer to :ref:`really-complicated-documents`) * an internal Ansar type. The container types and the object reference type require parameters that include further instances of type expressions. This is a recursive definition that allows for expression of arbitrarily complex types. Basic Python Types ------------------ The Python types that are accepted in type expressions, along with their respective Ansar equivalents, are listed below. Internally all Python types are converted to their Ansar equivalents before use. Either set of types can be used. Using the Python types is probably more Pythonic. +---------------+--------------+------------------------------------+ | Python | Ansar | Notes | +===============+==============+====================================+ | ``bool`` | ``Boolean`` | True or false. | +---------------+--------------+------------------------------------+ | ``int`` | ``Integer8`` | A large, signed integer. | +---------------+--------------+------------------------------------+ | ``float`` | ``Float8`` | A large, floating-point number. | +---------------+--------------+------------------------------------+ | ``bytearray`` | ``Block`` | A sequence of `arbitrary bytes`. | +---------------+--------------+------------------------------------+ | ``bytes`` | ``String`` | A sequence of `printable bytes`. | +---------------+--------------+------------------------------------+ | ``str`` | ``Unicode`` | A sequence of Unicode code-points. | +---------------+--------------+------------------------------------+ | ``uuid.UUID`` | ``UUID`` | A UUID from the standard Python | | | | library. | +---------------+--------------+------------------------------------+ Refer to the following section for more information, especially relating to the significance of the different "string" types. .. _ansar-types: The Ansar Types --------------- Full set of Ansar types appears below. +-------------------------+---------------+-------------------------------------+ | Ansar | Python | Notes | +=========================+===============+=====================================+ | Boolean | ``bool`` | True or false. | +-------------------------+---------------+-------------------------------------+ | Byte | ``int`` | A single `arbitrary byte`. | +-------------------------+---------------+-------------------------------------+ | Character | ``bytes`` | A single `printable byte`. | +-------------------------+---------------+-------------------------------------+ | Rune | ``str`` | A single Unicode code-point. | +-------------------------+---------------+-------------------------------------+ | Integer | ``int`` | Integer2/Integer4/Integer8. Signed | | | | integer numbers consuming 2-8 | | | | bytes. | +-------------------------+---------------+-------------------------------------+ | Unsigned | ``int`` | Unsigned2/Unsigned4/Unsigned8. | | | | Unsigned, integer numbers consuming | | | | 2-8 bytes. | +-------------------------+---------------+-------------------------------------+ | Float | ``float`` | Float4/Float8. Signed, floating- | | | | point numbers consuming 4 or 8 | | | | bytes. | +-------------------------+---------------+-------------------------------------+ | Enumeration | ``int`` | Names for operational numbers. | +-------------------------+---------------+-------------------------------------+ | Block | ``bytearray`` | A sequence of `arbitrary bytes`. | +-------------------------+---------------+-------------------------------------+ | String | ``bytes`` | A sequence of `printable bytes`. | +-------------------------+---------------+-------------------------------------+ | Unicode | ``str`` | A sequence of Unicode code-points. | +-------------------------+---------------+-------------------------------------+ | WorldTime | ``float`` | A Python time, i.e. ``float``. | +-------------------------+---------------+-------------------------------------+ | ClockTime | ``float`` | A Python time, i.e. ``float``. | +-------------------------+---------------+-------------------------------------+ | TimeSpan | ``float`` | A time delta, i.e. t2 - t1. | +-------------------------+---------------+-------------------------------------+ | UUID | ``UUID`` | A Python uuid.UUID. | +-------------------------+---------------+-------------------------------------+ | ArrayOf(`type`, `size`) | ``list`` | Fixed number of objects. | +-------------------------+---------------+-------------------------------------+ | VectorOf(`type`) | ``list`` | A sequence of zero or more objects. | +-------------------------+---------------+-------------------------------------+ | SetOf(`type`) | ``set`` | A collection of unique objects. | +-------------------------+---------------+-------------------------------------+ | MapOf(`key`, `type`) | ``dict`` | A collection of key-value pairs. | +-------------------------+---------------+-------------------------------------+ | DequeOf(`type`) | ``deque`` | A double-ended queue of objects. | +-------------------------+---------------+-------------------------------------+ | UserDefined(`type`) | | An instance of a registered class. | +-------------------------+---------------+-------------------------------------+ | Any | | An instance of any registered | | | | class. | +-------------------------+---------------+-------------------------------------+ | PointerTo(`type`) | | An object that may appear multiple | | | | times in the single representation. | +-------------------------+---------------+-------------------------------------+ | Type | ``class`` | A registered class. | +-------------------------+---------------+-------------------------------------+ | Word | | A generic form. | +-------------------------+---------------+-------------------------------------+ | Address | | Internal use. | +-------------------------+---------------+-------------------------------------+ | TargetAddress | | Internal use. | +-------------------------+---------------+-------------------------------------+ .. note:: The Ansar type system is a `portable` type system that must consider other programming languages and machine architectures. Further information on this topic is outside the scope of this document. Suffice to say that the library accepts the Basic Python Types precisely so that Python developers can work within the Python type system, as much as possible. .. _strings-of-things: Strings Of Things +++++++++++++++++ The ``Byte``, ``Character`` and ``Rune`` types facilitate the proper handling of an `arbitrary byte`, a `printable byte` and a Unicode code-point, respectively. There are no exact Python equivalents for these types as Python stores these values as "strings of length 1". They can be used in type expressions for finer control over the representation of those short strings. The ``Block``, ``String`` and ``Unicode`` types describe sequences of ``Byte``, ``Character`` and ``Rune``, respectively. The ``String`` and ``Block`` types result in different representations of the same application data, i.e. a sequence of bytes. The former assumes that there is a benefit to passing on the printable bytes (0x20 through to 0x7E) without alteration, i.e. for readability. The non-printing bytes will be "escaped" using the mechanism appropriate to the current encoding. The ``Block`` type is intended for the handling of binary data, such as the block-by-block transfer of image files. Blocks of bytes are always base64-encoded within representations. .. warning:: Application "string" variables containing values in the range of 0 to 255 are entirely valid. However, the specification for the XML encoding does not provide for the representation of all the values in that range. This affects use of the Python ``bytes`` type and the library ``Character`` and ``String`` types. Variables that contain exclusively "printable" values such as those values from decimal 32 (space) through to decimal 126 (tilde) are safe when using the XML encoding. Where variables may contain values outside that range, e.g. from decimal zero (nul) to decimal 31 (unit separator) and from decimal 127 (del) through to decimal 255, the behaviour of the XML encoding is not fully defined. There are differences relating to the versions of XML (the library generates exclusively XML 1.0 encodings) and the XML parser in use. Note that the library uses the standard Python XML parser for input, but that other tools in a development environment will include their own parser components. The ``bytes``, ``Character`` and ``String`` types are best reserved for "printable" content. If there is explicit need for non-printable content then applications should use the JSON encoding. Numbers And CPUs ++++++++++++++++ The ``Integer``, ``Unsigned`` and ``Float`` types facillitate the proper handling of the implied number types. The full names for these types include a suffix, e.g. ``Unsigned4``, which refers to the number of bytes consumed by that type. The Python type system manages numbers in a different manner, i.e. without the same regard to portability as Ansar. All type expressions involving numbers should use the Basic Python Types. The wider set of types offered by Ansar relate to the design of application data that must be portable across multiple programming languages and/or CPU architectures. By using the Basic Python Types, an application ensures that data is portable across Python implementations. Time Values +++++++++++ The ``WorldTime`` and ``ClockTime`` types provide for readable representations of standard Python time values (i.e. ``float``). A ``WorldTime`` is a GMT or Zulu representation. This is the "safest" representation in that it exhibits "pass-through" behaviour. The floating point value does not change even as the value is stored and recovered many times, potentially by different applications executing in different timezones. A ``WorldTime`` conforms to ISO 8601 and looks like this:: 2008-07-01T19:01:37Z A ``ClockTime`` uses the same representation, omitting the timezone specification (i.e. ``Z``). 2010-06-06T06:46:01.866233 Recovering a ``ClockTime`` produces a value appropriate to the local timezone. A ``ClockTime`` implements the concept of a specific time on a particular calendar day - that being a different ``float`` value for every timezone in the world. A ``TimeSpan`` is a relative value or `delta`, the difference between two time values. A few example representations appear below: * ``1h2m3s`` * ``8h`` * ``10m`` * ``0.0125s`` * ``1d`` These are readable text representations of `float` values. After recovery the resulting value can be added to any Python time value and that will have the expected effect, i.e. adding a recovered ``TimeSpan`` of ``2m`` will add ``120.0`` to the Python time value (noting that true time intervals are not nearly as simple as two time values and a bit of arithmetic):: >>> t = ar.clock_epoch('2012-03-06T19:00:30') >>> t 1331013630.0 >>> d = ar.text_to_span('2m') >>> d 120.0 >>> t += d >>> ar.clock_iso(t) '2012-03-06T19:02:30' Functions such as ``clock_epoch``, ``text_to_span`` and ``clock_iso`` are the functions used by the library to perform the translations needed, during store and recover operations. User Defined Objects ++++++++++++++++++++ The proper type expression for a registered class looks like:: ar.UserDefined(ReceivedJob) The shorthand form ``ReceivedJob`` is accepted. The expression is used to form the proper name-value pairs during storage and perform the proper member assignments during recovery. The registration of Python classes combined with the ``UserDefined`` type expression provides the basis for recovery of fully-realized application types. To express the concept of "any registered class", use the following:: ar.Any Refer to Polymorphic Persistence, in :ref:`more-about-types`. Object Pointers +++++++++++++++ The proper type expression for an object that may appear at multiple points in a single store operation, looks like:: ar.PointerTo(ReceivedJob) The library uses these "pointers" to manage a collection of shared objects, and to reconstruct references to those objects during a recover operation. Internal Ansar Types ++++++++++++++++++++ There are a small number of type expressions that exist for internal use. They have limited utility for an application. The ``Type`` and ``Word`` expressions are used in the implementation of both the ``Any`` feature and the ``Incognito`` class, the latter used to manage the recovery of unregistered classes. The ``Type`` expression could be used in the following manner:: >>> f = ar.File('registered-class', ar.Type) >>> f.store(ReceivedJob) >>> >>> r, _ = f.recover() >>> r .. note:: The parameter passed to the :py:meth:`~ansar.file.File.store` method is a Python `class` not an instance of that class. Notes On Usage And Implementation --------------------------------- The library needs type expressions to be precise, meticulous and complete. Sustaining those standards at all times is difficult, so the library takes an active role by resolving a few common mis-codings. Using Python Types is Okay ++++++++++++++++++++++++++ Internally the library implements an abstract type system that has broader requirements than only meeting the needs of a Python application. This can be confusing and where there are no broader intentions - e.g. portability of stored materials across languages and CPUs - any such confusion is counter-productive. For this reason the library supports the use of Python types within type expressions. It immediately translates instances of Python types into their library equivalents. There is no runtime difference to the host application. Type Expressions Are Values +++++++++++++++++++++++++++ There is a fairly subtle difference between what can seem to be a valid type expression and what a properly formed type expression actually needs to be. For this reason all type expressions passed to the library are put through a "clean-it-up" process that makes small adjustments. These are mostly to do with missing parentheses. Consider the type expression: .. code-block:: python te = ar.MapOf(ar.Byte, ar.ArrayOf(ar.WorldTime, 4)) This expression is more correctly written as: .. code-block:: python te = ar.MapOf(ar.Byte(), ar.ArrayOf(ar.WorldTime(), 4)) The ``MapOf`` and ``ArrayOf`` objects are expecting further `objects` as arguments, not the ``Byte`` and ``WorldTime`` `classes` that are being passed in the first version. These kinds of errors are insidious and fixable at that point where the expression is presented. Class Names Are Not Values ++++++++++++++++++++++++++ It is convenient and results in code clarity, but a registered class is not actually a type expression that can be used during operation of the library. Internally all references to registered classes are upgraded to instances of ``UserDefined``. This code: .. code-block:: python f = ar.File('job', ReceivedJob) Becomes this: .. code-block:: python f = ar.File('job', ar.UserDefined(ReceivedJob)) The library makes significant use of dispatching during its internal operations, based on an item of application data and its associated type expression. Dispatching is much more efficient if it is dealing with the single possibility of ``UserDefined`` rather than the list of all the classes that have been registered within the application. Classes And Methods =================== ``File`` -------- .. currentmodule:: ansar.file .. autoclass:: File :members: store, recover :member-order: bysource .. _file-exceptions: Exceptions raised by ``File`` +++++++++++++++++++++++++++++ .. autoclass:: FileFailure .. autoclass:: FileNotFound .. autoclass:: FileNoAccess .. autoclass:: FileNotAFile .. autoclass:: FileIoFailed .. autoclass:: FileEncoding Refer also to :ref:`Codec exceptions` ``Folder`` ---------- .. currentmodule:: ansar.folder .. autoclass:: Folder :members: folder, file, store, matching, each, recover, add, update, remove, clear, erase, exists, key, name :member-order: bysource Exceptions raised by ``Folder`` +++++++++++++++++++++++++++++++ Refer to :ref:`File exceptions` ``Codec`` --------- .. currentmodule:: ansar.json .. _codec-json: .. autoclass:: CodecJson .. currentmodule:: ansar.lmx .. autoclass:: CodecXml .. currentmodule:: ansar.codec .. _codec-base: .. autoclass:: Codec :members: encode, decode :member-order: bysource .. _codec-exceptions: Exceptions raised by ``Codec`` ++++++++++++++++++++++++++++++ .. autoclass:: CodecError .. autoclass:: CodecUsage .. autoclass:: CodecFailed ``Message And Bind`` -------------------- .. currentmodule:: ansar.message .. autoclass:: Message .. autoclass:: Unknown .. autoclass:: Incognito .. autofunction:: default_time .. autofunction:: default_uuid .. autofunction:: default_vector .. autofunction:: default_set .. autofunction:: default_map .. autofunction:: default_deque .. autofunction:: default_value .. currentmodule:: ansar.bind .. autofunction:: bind Exceptions raised by ``Message And Bind`` +++++++++++++++++++++++++++++++++++++++++ .. currentmodule:: ansar.message .. autoclass:: MessageError .. autoclass:: MessageRegistrationError ``Versioning`` -------------------- .. currentmodule:: ansar.version .. autofunction:: migrate