mmf.objects

attributes	Module to facilitate attribute definition.
InitializationError	Initialization failed or incomplete.
NameClashWarning	Name clash.
NameClashWarning1	Name clash.
NameClashWarning2	Name clash.
Archivable	Convenience class implementing
StateVars(*varargin, **kwargs)	Base class for objects that support definition of attributes through the specification of _state_vars class attributes and then processed by process_vars().
Container(*varargin, **kwargs)	Simple container class.
Attr	Base class for all attribute types.
Excluded(value)	Excluded variable.
Internal(value)	Internal variable.
Fast(value)	Fast variable.
Computed(*v, **kw)	Computed attributes.
ClassVar(value)	Class variables (not associated with instances).
NoCopy(value[, copy])	Default value for objects that should not be copied (or copied with a custom copier copy).
Delegate(cls[, vars, methods, value, cached])	Delegate Class.
Ref(ref[, cached, method])	Ref Class. Represents an attribute reference to another
attribute	Attribute class: like property, but with default operators.
parameter	Parameter class: Turns a function into a parameter.
calculate	Decorator for calculational functions that should be memoized.
MemoizedContainer(*argv, **kwargs)	The paradigm for this class is that one has a set of parameters (call them x[0], x[1] etc.) and a set of functions that depend on these parameters (call them f[0], f[1] etc.).
option	>>> class A(object):
Options([c])	Options class.
RecView	A simple subclass of the numpy recarray record array class that provides automatic conversion to a recarray.
Calculator(*argv, **kwargs)	This class represents some data and values that must be calculated by a function calculate.
process_vars(*args, **kw)	Return (results, wrap__init__).
class_NamedArray(array_[, names])	Return a class that is a subclass of array_.__class__ but that allows the elements to be accessed through a named member syntax.
recview(a, names[, formats, dtype])	Return a view of a as a recarray with defined field names.

Inheritance diagram for mmf.objects:

This file contains a set of classes and meta-classes that allow objects to inherit useful functionality dealing with attributes, automatic calculations etc.

Overview

The highlight of this module is StateVars. The provides a powerful method for providing and documenting attributes with mechanisms for specifying defaults and delegates.

Performance

Attention has been given to making getting attributes as efficient as if they were standard attributes. Setting attributes is not so efficient, however, as many checks are done at this stage. This reflects the common usage of attributes such as parameters that are set only once in a while but accessed frequently.

Note

Getting attributes is somewhat inefficient for references as lookup must generally be performed. This penalty can be overcome by allowing references to cache values in the local dictionary, but this can get out of sync if the referenced values are in a delegate and changed via the delegate.

StateVars

__init__() Semantics

In the StateVars and its descendants, the role of StateVars.__init__() and the inherited __init__() method changes slightly from the usual role as an initializer. The semantics are that of an initializer that ensures the class is in a consistent state after initialization and after variable changes. The following changes have been made:

1. StateVars.__setattr__() calls __init__() whenever a variable is assigned. The name of the change variable is passed as a keyword argument so that a clever __init__() can avoid performing unnecessary computations. (See also StateVars.suspend(), StateVars.resume(), and StateVars.initialize()).

   Warning

   During construction, all default values are also assigned, and these assignments are included as keyword arguments to __init__(). Thus, there is no way of knowing exactly what arguments were specified by the user. This is intentional so that code to compute Computed variables can simply check for all values that have been set – default or otherwise – and compute what is needed.

   The workaround is to set a special default value (such as None or defining a special object containing the value) so that you can differentiate from a significant value. Of course, if the user provides exactly this value, you are in the same boat. (Consider using a private class for example to prevent this.)

StateVars.__new__() also wraps the StateVars.__init__() method defined by the class to provide the following behaviour. (This is defined in _get_processed_vars().)

2. All assignments of named arguments to the instance attributes takes place before StateVars.__init__() is called, so this usual self.a = a statements may be omitted.
3. The base class __init__() methods are called if StateVars._call_base__init__ is True, otherwise the user should call them as usual.

Thus, the main purpose of __init__() is to compute all Computed variables from the changed (or initialized) values of the other class parameters. Here is a skeleton:

def __init__(self, *varargin, **kwargs):
    if (1 == len(varargin) and not kwargs):
        # Called as a copy constructor.  State has been copied from
        # other by __new__.  You do not need to copy the
        # attributes here, but may need to copy calculated
        # parameters that are cached.  If kwargs is not empty,
        # then additional values have been specified and need to
        # be processed, so we exclude that case here
        _other = varargin[0]
    elif 1 < len(varargin):
        # Another argument passed without kw.  Error here.
        raise ValueError("StateVars accepts only keyword arguments.")
    else:
        # Process the changed variables as specified in kwargs
        # Call the base __init__ methods unless you explicitly set
        # _call_base__init__ is set
        <BaseClass>.__init__(self, *varagin, **kwargs)

        # For all computed vars, if a dependent variable is in
        # kwargs, then recompute it.

Property Semantics

Properties can have three roles:

1. As a way of defining dynamically computed data. In this case the property should have no fset method and will be stored as a Computed state var with no storage.
2. As a way of getting and setting other attributes. These types of properties have fset but should not be archived, stored etc. The data will be archived and restored when the other data are stored. Such properties should not be included in _state_vars: They will instead be listed in the _settable_properties attribute.
3. As the preferred way of getting and setting other attributes in a way that the data should be stored and archived. In this case, the data attributes used by the property should be directly set in the dictionary. These properties should be included in _state_vars (which will generate a name clash error, so be sure to add the appropriate name to _ignore_name_clash).

The first two roles will be deduced automatically if a property exists but is not in _state_vars. The last role requires both the existence of the property and an entry in _state_vars.

Copy Semantics

Copying in python can be a slippery business, especially if one wants to maintain some shared data, or to use functions, method, or classes as data. First we define

1. “Controlled data” in a StateVars instance must be explicitly specified in the StateVars._state_vars attribute. This will be copied as specified by the user.
2. “Uncontrolled data” in a StateVars instance is that which is stored in the __dict__. This is copied as a dictionary would be copied with no special processing.

Todo

Revisit copying, especially deepcopy to remove the need for hacks.

In order to manage the copying and referencing of data we use the following semantics:

1. All of the real data is assumed to be stored in __dict__. To copy the object, thus, only this dictionary needs to be copied. The methods __copy__() does this, making sure that the delegates are copied while the other attributes are simply referenced. Method __deepcopy__() works as usual.
2. The method items() returns a list of the data required to construct the object. The object should be able constructed by passing all of these items to the constructor. Note that the constructor generally must be called in order to perform initializations. Computed and Excluded attributes will not be included here as they should be recomputed by the initialization routines, but references will be.

To Do

Todo

Add support for nicely printing: i.e. flags or something to indicate how the object should print.

Todo

Add Dependent like Required so that some values can be used, represented, etc. but are the result of calculations. Errors or warnings should be issued if the user tries to set these.

Todo

Complete _preprocess() to support optional syntax.

Todo

Provide more functionality for subclasses to change default values of the parent classes. For example, if a parent class delegates x=a.x, then the subclass has to include (‘a.x’, 5) in _state_vars to change the default. If, instead, the subclass includes (‘x’, 5), then this will create a new variable and the delegation will be broken. This should raise a warning. Perhaps add a new type Default so that one can do (‘x’, Default(5)). This will raise an error unless x is already a properly settable attribute in a parent, and will use the previous definition, including any delegation. A related issue that this would solve is that there is presently no way to provide defaults for non-stored attributes that provide both setters and getters.

Known Bugs

Todo

Multiple inheritance requires a call to process_vars() even if no new _state_vars are added. There is probably no solution without __metaclass__ trickery.

Todo

If a class is defined in a local scope such as a function, then the local variables will be added to the class __dict__. This is because of how process_vars() works, but I don’t know how to fix it. See the test test_local_scope(). Somehow the dic passed to the metaclass includes local variables...

Todo

StateVars.__iter__() uses hasattr, which calls getattr. If the user defines getters, then this is can be problematic.

Optional Syntax

Here is another possible syntax (not yet fully implemented, see _preprocess()). The idea here is to define simple state vars of the class that are of a special type, either one of the Required, Delegate etc. etc., or a tuple of length 2 with the second element of a special type. Then delete these and add the corresponding entries to _state_vars. For example, the following could be equivalent:

class A(StateVars):
    _state_vars = [
        'a',
        ('b', 0),
        ('x', 1, 'variable x'),
        ('y=x', 'variable y (delegates to y)'),
        ('z', Required, 'a required var'),
        ('c', ClassVar(4), 'a class var')
        ]
    process_vars()

class A(StateVars):
    a = Attr
    b = 0, Attr
    x = 1, Attr('variable x')
    y = 'x', Ref('variable y (delegates to y)')
    z = Required('a required var'),
    c = 4, ClassVar('a class var')
    process_vars()

Note

To do this, we should change the syntax of the various classes so that they can accept documentation as their first argument (this will be removed by _preprocess() and placed as the third element in the _state_vars tuple, but we need to allow for this here).

class mmf.objects.InitializationError

Bases: exceptions.Exception

Initialization failed or incomplete.

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

class mmf.objects.NameClashWarning

Bases: exceptions.UserWarning

Name clash. Name already defined.

The default is to ignore warnings of type NameClassWarning1 but show those of type NameClassWarning2 which tend to be more severe. If you want to ignore all such errors (after debugging!) so you can disable this using simplefilter(). See warnings for possible actions: the most common are ‘one’ or ‘ignore’.

You can also do this on a class by class basis by defining the class attribute _ignore_name_clash. This should be a set of the known name clashes.

Note

This may be emitted because of spurious variables added to the class __dict__ if the class was defined within a local scope.

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

default_action = 'once'

classmethod simplefilter(action=None)

Filter NameClashWarnings with specified action.

class mmf.objects.NameClashWarning1

Bases: mmf.objects.NameClashWarning

Name clash. Name already defined.

This is a less serious warning, for example, it is raised when default values are changed. It is by default disabled.

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

default_action = 'ignore'

class mmf.objects.NameClashWarning2

Bases: mmf.objects.NameClashWarning

Name clash. Name already defined.

This is a more serious warning about name-clashes that are indicative of bugs. It is by default enabled to warn once.

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

default_action = 'once'

class mmf.objects.Archivable

Bases: object

Convenience class implementing interfaces.IArchivable.

The user only needs to implement the function items().

Examples

>>> class A(Archivable):
...     def __init__(self, a, b):
...         self.data = [a, b]
...     def items(self):
...         return zip(('a', 'b'), self.data)
>>> a = A(1, 2)
>>> a
A(a=1, b=2)

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return a list [(name, obj)] of (name, object) pairs

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

archive(name)

Return a string representation that can be executed to restore the object.

Examples

(Note: We can’t actually do this because the class A cannot be imported from a module if it is defined at the interpreter):

class A(Archivable):
     def __init__(self, a):
         self.a = a
     def items(self):
         return [('a', self.a)]
a = A(6)            # Create an instance
s = a.archive('n')  # Get string archive
env = {}            # Define an environment
exec(s, env)        # Evaluate the string in env
o = env['n']        # Access the object from env
o.a

archive_1(env=None)

Return (rep, args, imports).

Defines a representation rep of the instance self where the instance can be reconstructed from the string rep evaluated in the context of dict args with the specified imports = list of (module, iname, uiname) where one has either “import module as uiname”, “from module import iname” or “from module import iname as uiname”.

items()

Return a list [(name, obj)] of (name, object) pairs where the instance can be constructed as ClassName(name1=obj1, name2=obj2, ...).

The default implementation just uses the elements in self.__dict__.

class mmf.objects.StateVars(*varargin, **kwargs)

Bases: mmf.objects.Archivable

Base class for objects that support definition of attributes through the specification of _state_vars class attributes and then processed by process_vars(). For example:

class Complex(StateVars):
    "A complex number class."
    _state_vars = [
        ('real', 0, 'Real part'),
        ('imag', 0, 'Imaginary Real part')]
    process_vars()
    ...

When defining a StateVars class one should:

1. Define the _state_vars attribute as discussed below.

2. Call process_vars().

3. Define __init__() to specifying initialization. Be sure to call __init__() for any base classes (not needed if inheriting directly from StateVars. Remember that you do not need to assign any parameters in _state_vars — these will be assigned in the constructor __new__(). However, any Computed attributes will need to be computed here, and one might like to perform checks of the parameter values.

   Remember that __init__() will be called whenever parameter change in a shallow way (i.e. through direct assignment).

   See __init__() Semantics for details.

4. Profile performance: if performance is too slow then:

   1. Define _nodeps. This is a list of names that are simply used but which do not affect any computations in __init__(). Changes to these parameters will no longer trigger a call to __init__().
   2. Inspect the kwargs argument to __init__(). This will include all changed parameters so you can optimize the re-initialization to only perform the required updates.
   3. Consider calling suspend() and resume() in your code so that changes can be accumulated and processed by a single call to __init__() when resume() is called rather than when each attribute is changed.

The call to process_vars() will parse _state_vars from the class definition and any superclasses. These are accumulated and described in the following class attributes:

Notes

The state variables with names begining with underscores ‘_...’ are always excluded unless explicitly included in _state_vars.

_state_vars may be a dictionary with default values, or simply a list/set of variable names. See below for examples.

We also allow for parameters to delegate to other parameters by attributes with the following forms:

('x', Ref('y'), <doc>):

The object will have an attribute x that refers to the attribute y. The documentation may be changed, but the default value must be specified through y.

('a', Delegate(A), <doc>):

The attribute a will be initialized as an instance of class A taking it’s attributes from the constructor. All attributes with the name a.* will be used to provide keyword arguments. In addition, any other attributes of class A will be promoted to become attributes of the current class. It is an error if they collide with any other attributes.

('a', Delegate(A, ['x', 'y']), <doc>):

Same as above except only the attributes x and y will be promoted.

('a.y', <default>, ''):

This is a direct reference to the attribute a.y. If a is delegated from a class as above, then this can be used to provide a (new) default value. Any documentation is ignored here and this parameter is not listed. Instead, the default is listed in the documentation of the parameter a itself.

These are the normalized forms. Shortcuts are:

('x=a.y', <default>, <doc>):

Equivalent to the pair ('x', Ref('a.y'), <doc>) and ('a.y', <default>, '')

('x=y'):

Synonym for ('x', Ref('y'), _NO_DESCRIPTION)

('x=a.y', <default>):

Synonym for ('x=a.y', <default>, _NO_DESCRIPTION)

Examples

>>> class A(StateVars):
...     _state_vars = [('x', 10), ('y', 100)]
...     process_vars()
>>> a = A(x=1.0)
>>> a
A(x=1.0)
>>> print a
x=1.0
y=100
>>> a.y = 5
>>> a
A(x=1.0, y=5)

Setting other attributes is not transparently supported...

>>> a.z = 6
Traceback (most recent call last):
    ...
AttributeError: Cannot set attribute 'z' of object 'A'

You can if you must, however...

>>> a.__dict__['z'] = 6
>>> a.z
6

But direct assignment is still not supported

>>> a.z = 7
Traceback (most recent call last):
    ...
AttributeError: Cannot set attribute 'z' of object 'A'

If you forget to call process_vars()...

>>> class B(A):
...     _state_vars = [('LargeObject', Excluded)]

A helpful reminder is displayed upon use...

>>> b = B()
Traceback (most recent call last):
   ...
InitializationError: 'B' class unprocessed.
    Please call process_vars() in class definition.

The idea of Excluded parameters is to store large temporary calculations for inspection. These should never be used within the class, and production code should function without these objects. They are intended for debugging, or for presenting users with intermediate results. Final results should be returned as different persistent structures.

>>> class B(A):
...     _state_vars = [('LargeObject', Excluded)]
...     process_vars()
>>> b = B(x=2)
>>> print b
x=2
y=100
>>> b.LargeObject = 'Elephant'
>>> b
B(x=2)
>>> print b
x=2
y=100
Excluded:
LargeObject=Elephant

The __init__() method should do any lengthy calculations.

>>> class C(StateVars):
...     _state_vars = ['x', 'y',
...                    ('x2', Computed),
...                    ('y2', Computed),
...                    ('z', 0),
...                    ('t', 0),
...                    ('e', Excluded(0))]
...     _nodeps = ['z']
...     process_vars()
...     def __init__(self, **kwargs):
...         print "Initializing"
...         if 'x' in kwargs:
...             print "Calculating x^2"
...             self.x2 = self.x**2
...         if 'y' in kwargs:
...             print "Calculating y^2"
...             self.y2 = self.y**2
>>> c = C(x=1, y=2)
Initializing
Calculating x^2
Calculating y^2
>>> print c
x=1
y=2
z=0
t=0
Computed:
x2=1
y2=4
Excluded:
e=0

Note that __init__() is also responsible for recomputing values if the inputs change. If the calculations are tedious, then a controlled computation like that shown should be used to reduce the work.

Note

This recomputing only works for attributes that are assigned, i.e. iff __setattr__() is called. If an attribute is mutated, then __getattr__() is called and there is no (efficient) way to determine if it was mutated.

(Future versions might consider tagging all “accessed” attributes, however, in order to determine if a mutable object was changed, one would need to store a copy, which could provide a bad performance hit).

>>> c.x = 3
Initializing
Calculating x^2
>>> print c
x=3
y=2
z=0
t=0
Computed:
x2=9
y2=4
Excluded:
e=0
>>> c.t = 6
Initializing
>>> c.z = 7
>>> c.e = 9

Note that __init__() was not called when z was assigned because it is included in the _nodeps list or when e was assigned because it is Excluded. Calls to __init__() can be deferred by using suspend() and resume():

>>> c.suspend()
>>> c.x = 4
>>> c.y = 6
>>> c.resume()
Initializing
Calculating x^2
Calculating y^2
>>> print c
x=4
y=6
z=7
t=6
Computed:
x2=16
y2=36
Excluded:
e=9

Note, you may use properties. If these are not included in _state_vars, then these will be added to _state_vars by process_vars() either as un-stored Computed attributes or as _settable_properties. Neither will be archived or returned by items().

>>> class E1(StateVars):
...     _state_vars = ['y']
...     process_vars()
...     @property
...     def x(self):
...         '''Another name for 2*y'''
...         print "Getting x=2*y"
...         return 2*self.y
>>> a = E1(y=2) # x is not set but getattr is called by hasattr...
Getting x=2*y
>>> a           # x is not set but getattr is called by hasattr...
Getting x=2*y
E1(y=2)
>>> a.x = 6
Traceback (most recent call last):
   ...
AttributeError: Cannot set attribute 'x' of object 'E1'

Here is an example where the property is settable (but still will not be archived.)

>>> class E2(StateVars):
...     _state_vars = ['y']
...     process_vars()
...     def set_x(self, x):
...         '''Another name for `2*y`'''
...         self.y = x/2
...         print "Setting x=2*y"
...     def get_x(self):
...         print "Getting x=2*y"
...         return 2*self.y
...     x = property(get_x, set_x)
>>> a = E2(y=2)
>>> a
E2(y=2)
>>> a.x = 6
Setting x=2*y
>>> print a
y=3.0

If you want the property to represent stored data, you must explicitly include it in _state_vars and suppress the name clash warning:

>>> class F(StateVars):
...     _ignore_name_clash = set(['x'])
...     _state_vars = ['y', 'x']
...     process_vars()
...     def set_x(self, x):
...         '''A stored property'''
...         self.__dict__['x'] = x
...         print "Setting x"
...     def get_x(self):
...         print "Getting x"
...         return self.__dict__['x']
...     x = property(get_x, set_x)
>>> a = F(y=2)
Getting x
>>> a  # Even though x is not set, getattr is called by hasattr...
Getting x
F(y=2)
>>> a.x = 5
Setting x
>>> print a
Getting x
Getting x
y=2
x=5

If no setter is defined, the property will be included as a Computed value.

Note

Again, there are some extra Getting calls as hasattr() is called to check if the attribute exists.

>>> class F(StateVars):
...     _ignore_name_clash = set(['y'])
...     _state_vars = [
...         ('y', 5, 'This property has a default and is stored')]
...     process_vars()
...     @property
...     def x(self):
...         '''property `x=2*y`'''
...         print "Getting x"
...         return self.y*2
...     def set_y(self, y):
...         self.__dict__['y'] = y
...         print "Setting y"
...     def get_y(self):
...         print "Getting y"
...         return self.__dict__['y']
...     y = property(get_y, set_y)
>>> a = F()     # x is not set but getattr is called by hasattr...
Getting y
Setting y
Getting y
Getting x
Getting y
>>> a
Getting y
Getting x
Getting y
Getting y
Getting y
Getting y
F()
>>> print a
Getting y
Getting y
Getting x
Getting y
Getting x
Getting y
y=5
Computed:
x=10

Class variables may also be specified. This provides a way of documenting and requiring class-specific variables. These variables are not archived or printed, but are documented in the class. They act as “class constants”.

>>> class F(StateVars):
...     _state_vars = [('A', ClassVar(Required), "Required 'class' var"),
...                    ('B', ClassVar(1), "Optional class var"),
...                    ('a', 2, "Regular parameter")]
...     process_vars()
>>> help(F)             
Help on class F...

class F(StateVars)
 |  Usage: F(a=2)
 |
 |  **Class Variables:**
 |      A: Required 'class' var
 |      B: Optional class var
 |  
 |  **State Variables:**
 |      a: Regular parameter
 |  
 |  Method resolution order:
...
 |  Data and other attributes defined here:
 |  
 |  A = Required
...
 |
 |  B = 1
...
>>> print F.__doc__
Usage: F(a=2)

Attributes
----------

**Class Variables:**
    A: Required 'class' var
    B: Optional class var

**State Variables:**
    a: Regular parameter
>>> f = F()
Traceback (most recent call last):
    ...
AttributeError: Required class variable 'A' not initialized in class 'F'
>>> class G(F):
...     _ignore_name_clash = set(['A', 'B'])
...     _state_vars = [('A', ClassVar(2), 'No longer required'),
...                    ('B', ClassVar(2))]
...     process_vars()
>>> print G.__doc__
Usage: G(a=2)
    
Attributes
----------

**Class Variables:**
    A: No longer required
    B: Optional class var

**State Variables:**
    a: Regular parameter
>>> g = G()
>>> g
G()
>>> print g
a=2
>>> g.A, g.B
(2, 2)

If a class variable is specified, it may later be replaced by an instance variable through inheritance. Again, be sure to ignore the name clash warning if this is deliberate.

>>> class H(F):
...     _ignore_name_clash = set(['A'])
...     _state_vars = [('A', Required)]
...     process_vars()
>>> print H.__doc__
Usage: H(A=Required, a=2)

Attributes
----------
    
**Class Variables:**
    B: Optional class var

**State Variables:**
    A: Required 'class' var
    a: Regular parameter
>>> H(a=2)
Traceback (most recent call last):
    ...
ValueError: Required attribute 'A' not defined.

You may delegate variables to a member variable. By default delegation will allow arguments for the delegate to be passed to the delegate from the main object. First we create two classes A and B that we will use as delegates, then a class C that delegates to these. For class A will delegate all attributes from the class directly using the Delegate specifier with the class as an argument. This will specify that the object should have an instance a constructed from parameters passed directly to C. These will all be mirrored with parameters in C that dispatch directly to the instance a.

>>> class A(StateVars):
...     _state_vars = [('x', 1, 'var x')]
...     process_vars()

>>> class B(StateVars):
...     _state_vars = [('t', 1, 'var t'),
...                    ('z', 2, 'var z')]
...     process_vars()

>>> class C(StateVars):
...     _state_vars = [
...         ('a', Delegate(A)), # This will be constructed by __new__
...         ('b', B()),         # Previously constructed object
...         ('y=a.x', 10),      # Ref called y to a.x with new default
...         ('b.z', 3)]         # Change of value upon construction.
...     process_vars()

>>> c = C()
>>> c.a
A(x=10)
>>> c.b
B(z=3)

An example of delegating a required variable:

>>> class A(StateVars):
...     _state_vars = [('x=b.x', Required),
...                    ('b', Container())]
...     process_vars()
>>> a = A(x=[]); a
A(b=Container(x=[]))
>>> a.x.append(3); a
A(b=Container(x=[3]))

Note how reassignment works:

>>> A(x=[4], b=Container(x=[3]))    
A(b=Container(x=[4]))

Note

This is not a good idea! Please ignore... It is possible, but fraught with danger (it will break repr for example!)

This is best resolved by either making the references Computed attributes (read-only) or excluding the referenced object and making it private.

>>> class A(StateVars):
...     _state_vars = [('x=_b.x', Required),
...                    ('_b', Excluded(Container()))]
...     process_vars()
>>> A(x=[])                         
A(x=[])

Todo

Think about this. This is not a good solution! Perhaps one should allow delegate objects that are hidden so that the class controls them properly with construction etc.

Delegation also ensures that a new copy of the object will be initialized upon copy construction rather than just linked.

>>> class A(StateVars):
...     pass
>>> class B(StateVars):
...     _state_vars = [('a', Delegate(A))]
...     process_vars()
>>> b1 = B()
>>> b2 = B(b1)
>>> id(b1.a) == id(b2.a)
False

In order to avoid copy construction from copying objects, they should have the value NotImplemented:

>>> class A(StateVars):
...     _state_vars = [('c', 1)]
...     process_vars()
>>> a = A()
>>> a.c = None
>>> b = A(a)
>>> print b.c
None
>>> a.c = NotImplemented
>>> b = A(a)
>>> b.c                             
1

Note that the value here is the default: the NotImplemented value of a.c is ignored.

When inheriting from classes that define __init__(), one must make sure that the base class versions are called, either by setting _call_base__init__ or explicitly. (Note that StateVars only has a dummy __init__() method, so if you only inherit from this class, or from descendants that do not redefine __init__(), then you may omit this call, but it is considered bad form.)

>>> class A(StateVars):
...     _state_vars = [
...         ('a', Required),
...         ('c_a', Computed)]
...     process_vars()
...     def __init__(self, *v, **kw):
...         print('Computing c_a...')
...         self.c_a = self.a*2
>>> class B_bad(A):
...     _state_vars = [
...         ('c_b', Computed)]
...     process_vars()
...     def __init__(self, *v, **kw):
...         print('Computing c_b...')
...         self.c_b = self.a*3
>>> a = A(a=1)
Computing c_a...
>>> b = B_bad(a=1)           # This fails to call base constructor
Computing c_b...
>>> b.c_a
Traceback (most recent call last):
    ...
AttributeError: 'B_bad' object has no attribute 'c_a'

Here are two correct ways:

>>> class B1(A):
...     _state_vars = [
...         ('c_b', Computed)]
...     process_vars()
...     def __init__(self, *v, **kw):
...         A.__init__(self, *v, **kw)
...         print('Computing c_b...')
...         self.c_b = self.a*3

>>> class B2(A):
...     _state_vars = [
...         ('c_b', Computed)]
...     _call_base__init__ = True
...     process_vars()
...     def __init__(self, *v, **kw):
...         print('Computing c_b...')
...         self.c_b = self.a*3

>>> b = B1(a=1)
Computing c_a...
Computing c_b...
>>> b.c_a, b.c_b
(2, 3)
>>> b = B2(a=1)
Computing c_a...
Computing c_b...
>>> b.c_a, b.c_b
(2, 3)

StateVar object know how to be archived. One can modify this by defining two hooks: _pre_hook_archive() and _post_hook_archive(). These will be called before and after the call to items() that is used to archive the object. One common use of these is to convert a large list (useful for concatenation) to a single array to be stored using HD5 for example.

>>> import numpy as np
>>> class A(StateVars):
...     _state_vars = [('x', [])]
...     process_vars()
...     def __init__(self, *v, **kw):
...         if 'x' in kw and isinstance(self.x, np.ndarray):
...             self.x = [_a for _a in self.x]
...     def _pre_hook_archive(self):
...         self.__dict__['x'] = np.asarray(self.x)
...     def _post_hook_archive(self):
...         self.__init__(x=self.x)
>>> a = A(x=[np.ones((2,1)) for _n in xrange(5)])
>>> a.x
[array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]])]
>>> a._pre_hook_archive()
>>> a.x
array([[[ 1.],
        [ 1.]],

       [[ 1.],
        [ 1.]],

       [[ 1.],
        [ 1.]],

       [[ 1.],
        [ 1.]],

       [[ 1.],
        [ 1.]]])
>>> a._post_hook_archive()
>>> a.x
[array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]]), array([[ 1.],
       [ 1.]])]

Attributes

_state_vars	list() -> new empty list
_nodeps	set() -> new empty set object
_vars	list() -> new empty list
_defaults
_class_vars	set() -> new empty set object
_computed_vars
_excluded_vars	set() -> new empty set object
_method_vars	set() -> new empty set object
_dynamic	bool(x) -> bool
_refs
_irefs
_delegates
_settable_properties	set() -> new empty set object
_call_base__init__	bool(x) -> bool

_original__init__

meth

This is the original __init__() function is stored here. After the class is finished, the original __init__() is replaced by a wrapper (defined in _get_processed_vars()) that sets the variables and then calls this (as well as the base constructors if _call_base__init__ is True.)

Methods

all_items()	Return a list [(name, obj)] of (name, object) pairs containing all items available.
archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
initialize(**kwargs)	Calls __init__() passing all assigned attributes.
items([archive])	Return a list [(name, obj)] of (name, object) pairs where the
resume()	Resume calls to __init__() from __setattr__(),
suspend()	Suspend calls to __init__() from

This is the initialization routine. It is called after the attributes specified in _state_vars have been assigned.

The user version should perform any after-assignment initializations that need to be performed.

Note

Since all of the initial arguments are listed in kwargs, this can be used as a list of attributes that have “changed” allowing the initialization to be streamlined. The default __setattr__() method will call __init__() with the changed attribute to allow for recalculation.

This recomputing only works for attributes that are assigned, i.e. iff __setattr__() is called. If an attribute is mutated, then __getattr__() is called and there is no (efficient) way to determine if it was mutated.

Computed values with True Computed.save will be passed in through kwargs when objects are restored from an archive. These parameters in general need not be recomputed, but this opens the possibility for an inconsistent state upon construction if a user inadvertently provides these parameters. Note that the parameters still cannot be set directly.

See __init__() Semantics for details.

Methods

all_items()	Return a list [(name, obj)] of (name, object) pairs containing all items available.
archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
initialize(**kwargs)	Calls __init__() passing all assigned attributes.
items([archive])	Return a list [(name, obj)] of (name, object) pairs where the
resume()	Resume calls to __init__() from __setattr__(),
suspend()	Suspend calls to __init__() from

__init__(*varargin, **kwargs)

This is the initialization routine. It is called after the attributes specified in _state_vars have been assigned.

The user version should perform any after-assignment initializations that need to be performed.

Note

Since all of the initial arguments are listed in kwargs, this can be used as a list of attributes that have “changed” allowing the initialization to be streamlined. The default __setattr__() method will call __init__() with the changed attribute to allow for recalculation.

This recomputing only works for attributes that are assigned, i.e. iff __setattr__() is called. If an attribute is mutated, then __getattr__() is called and there is no (efficient) way to determine if it was mutated.

Computed values with True Computed.save will be passed in through kwargs when objects are restored from an archive. These parameters in general need not be recomputed, but this opens the possibility for an inconsistent state upon construction if a user inadvertently provides these parameters. Note that the parameters still cannot be set directly.

See __init__() Semantics for details.

all_items()

Return a list [(name, obj)] of (name, object) pairs containing all items available. This will include computed vars, reference, etc. and will likely be redundant.

See also

items()

archive_1(env=None)

Return (rep, args, imports).

Same as Archivable.archive_1() except passes archive=True to items().

classmethod class_keys()

Return the keys accepted by the constructor.

>>> StateVars.class_keys()
[]
>>> class A(StateVars):
...     _state_vars = ['a', 'b']
...     process_vars()
>>> A.class_keys()
['a', 'b']

classmethod get_args(*varargin, **kwargs)

Return the dictionary that is a subset of kwargs containing the valid keyword arguments for the current object.

>>> class A(StateVars):
...     _state_vars = ['a', 'b']
...     process_vars()
>>> class B(StateVars):
...     _state_vars = ['c', 'd']
...     process_vars()
>>> def f(**kwargs):
...     a = A(A.get_args(kwargs))
...     b = B(B.get_args(kwargs))
...     return a, b
>>> a, b = f(a=1, b=2, c=3, d=4)
>>> a, b
(A(a=1, b=2), B(c=3, d=4))

>>> Container(Container.get_args(a=1, b=2))
Container(a=1, b=2)

classmethod getargspec()

Get the names and default values of the constructor.

A tuple of four things is returned: (args, varargs, varkw, defaults). ‘args’ is a list of the argument names (it may contain nested lists). ‘varargs’ and ‘varkw’ are the names of the * and ** arguments or None. ‘defaults’ is an n-tuple of the default values of the last n arguments.

>>> StateVars.getargspec()
([], None, None, ())
>>> Container.getargspec()
([], None, 'kwargs', ())
>>> class D(StateVars):
...     _state_vars = [('a', 0), ('b', Required), ('c', NotImplemented)]
...     process_vars()
>>> D.getargspec()
(['b', 'a', 'c'], None, None, (0, NotImplemented))

initialize(**kwargs)

Calls __init__() passing all assigned attributes. This should not typically be needed as the use of suspend and resume should function appropriately. This can be used to set a bunch of parameters, however.

Parameters :

<name> : value

Set the parameter name to value through the keyword argument mechanism. If no parameters are provided, all variables returned by items() will be reassigned.

Examples

>>> class A(StateVars):
...     _state_vars = ['a', ('x', Computed)]
...     process_vars()
...     def __init__(self, a=None):
...         if a is None:
...             print "Initializing: a not changed."
...         else:
...             print "Initializing: a changed."
...             self.x = 2*a
>>> a = A(a=1); a.x
Initializing: a changed.
2
>>> a.a = 2; a.x
Initializing: a changed.
4
>>> a.suspend()
>>> a.a = 3
>>> a.resume(); a.x
Initializing: a changed.
6
>>> a.initialize();
Initializing: a not changed.
>>> a.initialize(a=8); a.x
Initializing: a changed.
16

items(archive=False)

Return a list [(name, obj)] of (name, object) pairs where the instance can be constructed as ClassName(name1=obj1, name2=obj2, ...).

Parameters :

archive : False, True, optional

If provided, then include Computed objects if they have their Computed.save flag is True.

resume()

Resume calls to __init__() from __setattr__(), first calling __init__() with all the cached changes.

suspend()

Suspend calls to __init__() from __setattr__(), but keep track of the changes.

class mmf.objects.Container(*varargin, **kwargs)

Bases: mmf.objects.StateVars

Simple container class. This is just a StateVars class with allowed dynamic allocation and with __getitem__() and __setitem__() mapped to __getattr__() and __setattr__() so that it behaves like a dictionary.

Examples

>>> a = Container(a=1, b=2)
>>> print a
a=1
b=2
>>> a.z = 4                 # Can define new attributes
>>> a.z
4
>>> a['b'] = 3              # Can use dictionary syntax
>>> print a
a=1
b=3
z=4

We also include an update method that updates the dictionary

>>> a.update(Container(b=3, c=4))
>>> print a
a=1
c=4
b=3
z=4

If you specify _dynamic in the constructor, then the Container will no longer accept new variable definitions:

>>> c = Container(a=1, b=2, _dynamic=False)
>>> print c
a=1
b=2
>>> c.z = 4
Traceback (most recent call last):
    ...
AttributeError: Cannot set attribute 'z' of object 'Container'

Methods

all_items()	Return a list [(name, obj)] of (name, object) pairs containing all items available.
archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
initialize(**kwargs)	Calls __init__() passing all assigned attributes.
items([archive])	Return a list [(name, obj)] of (name, object) pairs where the
resume()	Resume calls to __init__() from __setattr__(),
suspend()	Suspend calls to __init__() from
update(container)	Update the items from container.

This is the initialization routine. It is called after the attributes specified in _state_vars have been assigned.

The user version should perform any after-assignment initializations that need to be performed.

Note

Since all of the initial arguments are listed in kwargs, this can be used as a list of attributes that have “changed” allowing the initialization to be streamlined. The default __setattr__() method will call __init__() with the changed attribute to allow for recalculation.

This recomputing only works for attributes that are assigned, i.e. iff __setattr__() is called. If an attribute is mutated, then __getattr__() is called and there is no (efficient) way to determine if it was mutated.

Computed values with True Computed.save will be passed in through kwargs when objects are restored from an archive. These parameters in general need not be recomputed, but this opens the possibility for an inconsistent state upon construction if a user inadvertently provides these parameters. Note that the parameters still cannot be set directly.

See __init__() Semantics for details.

Methods

all_items()	Return a list [(name, obj)] of (name, object) pairs containing all items available.
archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
initialize(**kwargs)	Calls __init__() passing all assigned attributes.
items([archive])	Return a list [(name, obj)] of (name, object) pairs where the
resume()	Resume calls to __init__() from __setattr__(),
suspend()	Suspend calls to __init__() from
update(container)	Update the items from container.

__init__(*varargin, **kwargs)

This is the initialization routine. It is called after the attributes specified in _state_vars have been assigned.

The user version should perform any after-assignment initializations that need to be performed.

Note

Since all of the initial arguments are listed in kwargs, this can be used as a list of attributes that have “changed” allowing the initialization to be streamlined. The default __setattr__() method will call __init__() with the changed attribute to allow for recalculation.

This recomputing only works for attributes that are assigned, i.e. iff __setattr__() is called. If an attribute is mutated, then __getattr__() is called and there is no (efficient) way to determine if it was mutated.

Computed values with True Computed.save will be passed in through kwargs when objects are restored from an archive. These parameters in general need not be recomputed, but this opens the possibility for an inconsistent state upon construction if a user inadvertently provides these parameters. Note that the parameters still cannot be set directly.

See __init__() Semantics for details.

update(container)

Update the items from container.

class mmf.objects.Attr

Bases: mmf.objects.Archivable

Base class for all attribute types.

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

items()

Return all items to make object archivable. The attribute names are determined by inspecting the arguments to the constructor __init__().

class mmf.objects.Excluded(value)

Bases: mmf.objects._objects.HasDefault

Excluded variable.

These are not archived and are intended for either flags to indicate that a particular method is executing, or for non-essential reporting of internal state/temporary data.

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(value)

class mmf.objects.Internal(value)

Bases: mmf.objects.Excluded

Internal variable.

These are not archived and are intended for internal use only. Functionally equivalent to Excluded attributes and are presently implemented as these (and reported as these).

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(value)

class mmf.objects.Fast(value)

Bases: mmf.objects._objects.HasDefault

Fast variable.

Like Excluded and Internal variables, these do not trigger a call to __init__() when they are set, but they are stored in the archive. They should be used for simple attributes like flags or counters. Make sure that nothing depends on the value though, otherwise the object might not be properly updated.

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(value)

class mmf.objects.Computed(*v, **kw)

Bases: mmf.objects.Attr

Computed attributes.

These are not assignable, and should be computed in __init__() from other parameters.

Parameters :

save : False, True, optional

If this is True, then the value will be archived (i.e. included in a call to StateVars.items()) and will be directly restored when the object is archived using the tools in mmf.archive. Upon construction of a restored object, these computed values will be provided in kwargs so that __init__() can refrain from performing long tedious calculations.

Note

This is dangerous because one must make sure that the proper data is restored — if any of the other dependent parameters change, then the object could be left in an inconsistent state.

Todo

As a precaution, StateVars should only permits this type of construction if all items returned by StateVars.items() are assigned. This is related to issue 16.

Examples

>>> Computed.save
False
>>> Computed(save=True)
Computed(save=True)
>>> Computed(2)
Traceback (most recent call last):
   ...
TypeError: Computed() has no default value. Did you mean ClassVar(2)?

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Our constructor breaks the auto finding of attributes

__init__(*v, **kw)

items()

Our constructor breaks the auto finding of attributes

save = False

class mmf.objects.ClassVar(value)

Bases: mmf.objects._objects.HasDefault

Class variables (not associated with instances).

Consequential class variables should always be defined (and/or redefined) through the _state_vars mechanism. Inconsequential class variables can bed defined normally but these will always be overridden by those defined through the _state_vars mechanism once process_vars() is called.

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(value)

class mmf.objects.NoCopy(value, copy=False)

Bases: object

Default value for objects that should not be copied (or copied with a custom copier copy). The default semantics are that initialization variables are copied with the copier supplied to process_vars() (deepcopy if not specified).

Examples

>>> import copy
>>> n = NoCopy(3); n
NoCopy(3)
>>> n.copy                  
<function <lambda> at ...>
>>> n = NoCopy(3, copy='copy'); n
NoCopy(3, copy='copy')
>>> n.copy is copy.copy
True
>>> n = NoCopy(3, copy='deepcopy'); n
NoCopy(3, copy='deepcopy')
>>> n.copy is copy.deepcopy
True
>>> n = NoCopy(3, copy=lambda x:x); n 
NoCopy(3, copy=<function <lambda> at ...>)
>>> n.copy                  
<function <lambda> at ...>

__init__(value, copy=False)

copy

Return the copier

class mmf.objects.Delegate(cls, vars=None, methods=None, value=NotImplemented, cached=False)

Bases: mmf.objects._objects.HasDefault

Delegate Class.

An attribute with default value Delegate(A) will be initialized as A(**kw) where the attributes in A._state_vars are used as keyword arguments. Delegate(A, [‘x’,’y’]) is the same except only the specified attributes will be used. Note that the delegate constructor must accept these attributes as keyword args.

Member functions of the delegate may also be delegated, so that Delegate(A, [‘x’,’y’], methods=[‘f’]) will make the method f available in the top-level class.

A default may also be specified, in which case this will be used to initialize the object using the standard copy semantics as specified by NoCopy.

Examples

This is the simplest delegation to A. All variables in A._state_vars become members of D1:

>>> class A(StateVars):
...     _state_vars = [('a', 1), ('b', 2)]
...     process_vars()

>>> class D1(StateVars):
...     _state_vars = [
...         ('A_', Delegate(A), 'Delegate all vars')
...         ]
...     process_vars()

>>> d1 = D1(a=3); print d1
a=3
b=2
A_=a=3
b=2

The attributes a and b are actually the attributes of A_:

>>> d1
D1(A_=A(a=3))

>>> d1.a=5; print d1
a=5
b=2
A_=a=5
b=2

Now we delegate to B but include an explicit list of delegates. This is the safer thing to do as it reduces the chances of name clashes:

>>> class B(StateVars):
...     _state_vars = [('c', 3), ('d', 4)]
...     process_vars()
...     def f(self):
...         return self.c + self.d

>>> class D2(StateVars):
...     _state_vars = [('B_', Delegate(B, ['c'], methods=['f']))]
...     process_vars()

>>> d2 = D2(c=7); print d2
c=7
B_=c=7
d=4

Note that we can also call f: >>> d2.f() 11

If you want to delegate to an object that does not use _state_vars, you must explicitly specify these. Note that the ones you specify in the delegate must be accepted by the constructor.

>>> class C(Archivable):
...     def __init__(self, x=1, y=2):
...         self.x = x
...         self.y = y

>>> class D3(StateVars):
...     _state_vars = [('C_', Delegate(C, ['x','y']))]
...     process_vars()

>>> d3 = D3(x=10); d3
D3(C_=C(y=2, x=10))

Note that the assignments are made after the attribute C_ is defined, so that the local kwargs overwrite the values passed in to C_:

>>> D3(x=12, y=2, C_=C(y=2, x=10))
D3(C_=C(y=2, x=12))

Notice that the delegation mechanism creates a new instance of C here, hence it is shown in the representation. (It is slightly redundant to have kwargs as well but this to show that they are indeed attributes.)

You can also specify a default in the class definition. You may also use NoCopy etc.

>>> c = C(x=-1,y=-2)
>>> class D4(StateVars):
...     _state_vars = [('C_', Delegate(C, ['x','y'],
...                                    value=NoCopy(c)))]
...     process_vars()

This is dangerous though, because you will modify the original (and then since the default has changed, the changes will not appear in the representation!) >>> d4 = D4(x=5); d4 D4() >>> c C(y=-2, x=5) >>> id(c) == id(d4.C_) True

Finally, in order to optimize attribute access, you can specify that the delegate should be cached. This means that the delegated attributes will be cached in the instance __dict__ for efficient retrieval.

Warning

This can break code because it means that if you change an attribute in the delegate directly, the cache will not be updated until initialize() is called or the corresponding attribute is set via the reference.

>>> Delegate(None, ['x', 'y'])
Delegate(vars=['x', 'y'], cls=None)
>>> Delegate(None)
Delegate(cls=None)
>>> Delegate(None, vars=['x'], value=4)
Delegate(value=4, vars=['x'], cls=None)

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(cls, vars=None, methods=None, value=NotImplemented, cached=False)

class mmf.objects.Ref(ref, cached=False, method=False)

Bases: mmf.objects.Attr

Ref Class. Represents an attribute reference to another attribute.

Examples

>>> Ref('x')
Ref(ref='x')
>>> Ref('x', cached=True)
Ref(cached=True, ref='x')

Methods

archive(name)	Return a string representation that can be executed to restore the object.
archive_1([env])	Return (rep, args, imports).
items()	Return all items to make object archivable.

__init__(ref, cached=False, method=False)

class mmf.objects.attribute(func=None, default=None, doc='')

Bases: property

Attribute class: like property, but with default operators.

One way to use is to directly make a property with documentation and a default value:

x = attribute(default=1.0, doc=’x-coordinate’)

Another way is as a decorator:

@attribute def x(self):

‘x-coordinate’ return 1.0

Examples

>>> class Data(object):
...     x = attribute(default=1.0, doc='x-coordinate')
...     @attribute
...     def y(self):
...         'y-coordinate'
...         return -1.0
>>> d = Data()
>>> d.x
1.0
>>> d.x = 4
>>> d.x
4
>>> d.y
-1.0
>>> d.y = 2.0
>>> d.y
2.0
>>> help(d)         
Help on Data in module ...:

class Data(__builtin__.object)
...
 |  x
 |      x-coordinate
 |  
 |  y
 |      y-coordinate

Methods

deleter	Descriptor to change the deleter on a property.
getter	Descriptor to change the getter on a property.
setter	Descriptor to change the setter on a property.

Initialization.

Methods

deleter	Descriptor to change the deleter on a property.
getter	Descriptor to change the getter on a property.
setter	Descriptor to change the setter on a property.

__init__(func=None, default=None, doc='')

Initialization.

class mmf.objects.parameter(f)

Bases: property

Parameter class: Turns a function into a parameter.

This has several benifits: 1) The docstring of the function describes the parameter. 2) The return value is used to initialize the parameter. 3) If the parameter is an instance of the class MemoizedContainer, then memoization is used and the signal _changed is sent when the parameter is modified.

>>> class Data(object):
...     @parameter
...     def x():
...         "x coordinate"  # Documentation
...         return 10.0     # Default value
>>> d = Data()
>>> d.x
10.0
>>> d.x = 6
>>> d.x
6

If you want fast parameter access, use the dictionary directly: >>> d.__dict__[‘x’] 6

Methods

deleter	Descriptor to change the deleter on a property.
getter	Descriptor to change the getter on a property.
setter	Descriptor to change the setter on a property.

__init__(f)

class mmf.objects.calculate(f)

Bases: property

Decorator for calculational functions that should be memoized. Memoization only happens with instances of MemoizedContainer.

>>> class Data(object):
...     @calculate
...     def f(x):
...         "The square of x"
...         return x**2
>>> d = Data()
>>> d.x = 6
>>> d.f
36

Methods

deleter	Descriptor to change the deleter on a property.
getter	Descriptor to change the getter on a property.
setter	Descriptor to change the setter on a property.

__init__(f)

class mmf.objects.MemoizedContainer(*argv, **kwargs)

Bases: dict

The paradigm for this class is that one has a set of parameters (call them x[0], x[1] etc.) and a set of functions that depend on these parameters (call them f[0], f[1] etc.). The idea is that the functions should compute the value only once and memoize it until the parameters that it depends on change.

To do this, define all the bare parameters using the parameter class. Then define all of the functions as standard functions with arguments that have the same name as the parameters. The argument list determines the dependencies. These should be defined as functions, and should not accept the parameter self. The actual member functions will be defined later by __new__.

If you would like to define helper functions that are not memoized, prefix them with an underscore.

>>> class Data1(MemoizedContainer):
...     @parameter
...     def x():
...         "x coordinate"  # Documentation
...         return 10.0     # Default value
...     @parameter
...     def y():
...         "y coordinate"
...         return 2.0
...     @parameter
...     def z():
...         "z coordinate"
...         return -2.0
...     @calculate
...     def f(x):
...         "The square of x"
...         print "Computing f("+`x`+")"
...         return x**2
...     @calculate
...     def g(x, f, y):
...         "Product of x*f*y = x**3*y"
...         print "Computing g("+`x`+", "+`f`+", "+`y`+")"
...         tmp = x*f*y
...         return tmp
>>> d = Data1(y=2.0)
>>> d.f                     # slow
Computing f(10.0)
100.0
>>> d.f                     # fast
100.0
>>> d.g                     # slow
Computing g(10.0, 100.0, 2.0)
2000.0
>>> d.g                     # fast
2000.0
>>> d.y = 3.0
>>> d.f                     # fast
100.0
>>> d.g                     # slow
Computing g(10.0, 100.0, 3.0)
3000.0
>>> d.g                     # fast
3000.0
>>> try: d.g = 5
... except AttributeError, err:
...     print err
can't set attribute
>>> d.dependencies
{'y': set(['g']), 'x': set(['g', 'f']), 'z': set([]), 'g': set([]), 'f': set(['g'])}

One can also use multiple inheritance to combine parameters:

>>> class Data2(MemoizedContainer):
...     @parameter
...     def a():
...         return -1.0     # Default value
...     @parameter
...     def b():
...         return 2.0
...     @calculate
...     def h(a):
...         print "Computing h("+`a`+")"
...         return a**3
>>> class Data3(Data1, Data2):
...     @parameter
...     def c():
...         return 2.0
...     @calculate
...     def F(c, f, g, h):
...         print "Computing F("+`c`+", "+`f`+", "+`g`+", "+`h`+")"
...         return c+f*g*h
>>> class Data(Data3):
...     pass
>>> d = Data()
>>> d.F
Computing f(10.0)
Computing g(10.0, 100.0, 2.0)
Computing h(-1.0)
Computing F(2.0, 100.0, 2000.0, -1.0)
-199998.0
>>> d.g
2000.0
>>> d.F
-199998.0

For faster bare parameter access, use the dictionary

>>> d.x                     # slow
10.0
>>> d['x']                  # fastest
10.0

Methods

clear	D.clear() -> None. Remove all items from D.
copy	D.copy() -> a shallow copy of D
fromkeys(...)	v defaults to None.
get	D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None.
has_key	D.has_key(k) -> True if D has a key k, else False
items	D.items() -> list of D’s (key, value) pairs, as 2-tuples
iteritems	D.iteritems() -> an iterator over the (key, value) items of D
iterkeys	D.iterkeys() -> an iterator over the keys of D
itervalues	D.itervalues() -> an iterator over the values of D
keys	D.keys() -> list of D’s keys
pop	D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
popitem	D.popitem() -> (k, v), remove and return some (key, value) pair as a
setdefault	D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
update	D.update([E, ]**F) -> None. Update D from dict/iterable E and F.
values	D.values() -> list of D’s values
viewitems	D.viewitems() -> a set-like object providing a view on D’s items
viewkeys	D.viewkeys() -> a set-like object providing a view on D’s keys
viewvalues	D.viewvalues() -> an object providing a view on D’s values

__init__(*argv, **kwargs)

class mmf.objects.option(func=None, doc=None, default=None)

Bases: mmf.objects.parameter

>>> class A(object):
...     x = option(doc='Description of member x',
...                default=0)
...     name = option(default='')
>>> a = A()
>>> a.x
0
>>> a.x = 1
>>> a.x
1
>>> a.x.__doc__
'Description of member x'
>>> a.name = "MyName"
>>> a.name.__doc__
'Undocumented Option'

Methods

deleter	Descriptor to change the deleter on a property.
getter	Descriptor to change the getter on a property.
setter	Descriptor to change the setter on a property.

__init__(func=None, doc=None, default=None)

class mmf.objects.Options(c=None, **kwargs)

Bases: object

Options class. This class acts as a container. It provides methods to display the members. One can also define member functions, but their names must start with an underscore: all non-underscored names are assumed to be ‘options’.

The values defined in the base class are used as defaults. They will be deepcopied for each instance. Note that this means recursive data should not be used for parameters.

Use in conjunction with the option decorator to provide documentation for the options.

Examples

Here are some examples of direct usage:

>>> d1 = Options(['x', 'y', 'cc.x', 'cc.y'])
>>> d1.x = 3
>>> d1['y'] = 4
>>> d1.cc.x = 5
>>> d1.cc.y = 6
>>> d1
Options({'y': 4, 'x': 3, 'cc': Options({'y': 6, 'x': 5})})
>>> print d1
opts=Options(['x', 'y', 'cc.x', 'cc.y'])
opts.x=3
opts.y=4
opts.cc.x=5
opts.cc.y=6

You may define options with keywords too:

>>> d2 = Options(x=3, y=4)
>>> d2
Options({'y': 4, 'x': 3})

Here is an example which defines a base class with defaults:

>>> class MyOpts(Options):
...   x = 1
...   y = 2
...   z = [2]               # This will get copied
>>> c1 = MyOpts()
>>> c2 = MyOpts()
>>> c2.z[0] = 3
>>> print c1                # Note that z does not change here
opts=MyOpts(['x', 'y', 'z'])
opts.x=1
opts.y=2
opts.z=[2]
>>> print c2                # It only changes here.
opts=MyOpts(['x', 'y', 'z'])
opts.x=1
opts.y=2
opts.z=[3]

Here is an example with the option decorator:

>>> class MyOpts(Options):
...     @option
...     def abs_tol():
...         'Absolute tolerance'
...         return 2e-16
...     
...     rel_tol = option(default=2e-8,
...                      doc='Relative tolerance')
>>> c1 = MyOpts()
>>> print c1
opts=MyOpts(['abs_tol', 'rel_tol'])
opts.abs_tol=2e-16
opts.rel_tol=2e-08

__init__(c=None, **kwargs)

class mmf.objects.RecView

Bases: numpy.core.records.recarray

A simple subclass of the numpy recarray record array class that provides automatic conversion to a recarray. To subclass, simply define _names as a list of the field names and _array as a default array.

>>> from numpy import array
>>> class Coord(RecView):
...     _array = array([1.0, 2.0])
...     _names = ['x', 'y']
>>> c = Coord()
>>> c.x
array([ 1.])

Methods

all([axis, out])	Returns True if all elements evaluate to True.
any([axis, out])	Returns True if any of the elements of a evaluate to True.
argmax([axis, out])	Return indices of the maximum values along the given axis.
argmin([axis, out])	Return indices of the minimum values along the given axis of a.
argsort([axis, kind, order])	Returns the indices that would sort this array.
astype(t)	Copy of the array, cast to a specified type.
byteswap(inplace)	Swap the bytes of the array elements
choose(choices[, out, mode])	Use an index array to construct a new array from a set of choices.
clip(a_min, a_max[, out])	Return an array whose values are limited to [a_min, a_max].
compress(condition[, axis, out])	Return selected slices of this array along given axis.
conj()	Complex-conjugate all elements.
conjugate()	Return the complex conjugate, element-wise.
copy([order])	Return a copy of the array.
cumprod([axis, dtype, out])	Return the cumulative product of the elements along the given axis.
cumsum([axis, dtype, out])	Return the cumulative sum of the elements along the given axis.
diagonal([offset, axis1, axis2])	Return specified diagonals.
dot(b[, out])	Dot product of two arrays.
dump(file)	Dump a pickle of the array to the specified file.
dumps()	Returns the pickle of the array as a string.
field(attr[, val])
fill(value)	Fill the array with a scalar value.
flatten([order])	Return a copy of the array collapsed into one dimension.
getfield(dtype, offset)	Returns a field of the given array as a certain type.
item(*args)	Copy an element of an array to a standard Python scalar and return it.
itemset(*args)	Insert scalar into an array (scalar is cast to array’s dtype, if possible)
max([axis, out])	Return the maximum along a given axis.
mean([axis, dtype, out])	Returns the average of the array elements along given axis.
min([axis, out])	Return the minimum along a given axis.
newbyteorder([new_order])	Return the array with the same data viewed with a different byte order.
nonzero()	Return the indices of the elements that are non-zero.
prod([axis, dtype, out])	Return the product of the array elements over the given axis
ptp([axis, out])	Peak to peak (maximum - minimum) value along a given axis.
put(indices, values[, mode])	Set a.flat[n] = values[n] for all n in indices.
ravel([order])	Return a flattened array.
repeat(repeats[, axis])	Repeat elements of an array.
reshape(shape[, order])	Returns an array containing the same data with a new shape.
resize(new_shape[, refcheck])	Change shape and size of array in-place.
round([decimals, out])	Return a with each element rounded to the given number of decimals.
searchsorted(v[, side])	Find indices where elements of v should be inserted in a to maintain order.
setasflat(arr)	Equivalent to a.flat = arr.flat, but is generally more efficient.
setfield(val, dtype[, offset])	Put a value into a specified place in a field defined by a data-type.
setflags([write, align, uic])	Set array flags WRITEABLE, ALIGNED, and UPDATEIFCOPY, respectively.
sort([axis, kind, order])	Sort an array, in-place.
squeeze()	Remove single-dimensional entries from the shape of a.
std([axis, dtype, out, ddof])	Returns the standard deviation of the array elements along given axis.
sum([axis, dtype, out])	Return the sum of the array elements over the given axis.
swapaxes(axis1, axis2)	Return a view of the array with axis1 and axis2 interchanged.
take(indices[, axis, out, mode])	Return an array formed from the elements of a at the given indices.
tofile(fid[, sep, format])	Write array to a file as text or binary (default).
tolist()	Return the array as a (possibly nested) list.
tostring([order])	Construct a Python string containing the raw data bytes in the array.
trace([offset, axis1, axis2, dtype, out])	Return the sum along diagonals of the array.
transpose(*axes)	Returns a view of the array with axes transposed.
var([axis, dtype, out, ddof])	Returns the variance of the array elements, along given axis.
view([dtype, type])

__init__()

x.__init__(...) initializes x; see help(type(x)) for signature

class mmf.objects.Calculator(*argv, **kwargs)

This class represents some data and values that must be calculated by a function calculate.

>>> class C(Calculator):
...     @parameter
...     def x(): return 1.0
...     @parameter
...     def y(): return 2.0
...     def calculate(self):
...         self.a = 2*self.x
...         self.b = self.a*self.x*self.y
>>> class D(Calculator):
...     @parameter
...     def z(): return 3.0
...     def calculate(self):
...         self.c = self.z**3
>>> class E(C, D):
...     def calculate(self):
...         C.calculate(self)
...         D.calculate(self)
>>> c = C()
>>> c.x = 2.0
>>> c.y = 3.0
>>> d = D(z=5.0)
>>> e = E(c, d)
>>> e.calculate()
>>> e.a
4.0
>>> e.b
24.0
>>> e.c
125.0
>>> e
E(y=3.0, x=2.0, z=5.0)

Methods

calculate()

calculate()

mmf.objects.process_vars(*args, **kw)

Return (results, wrap__init__). This is a class decorator to process the _state_vars attribute of cls and to define the documentation, and members _state_vars, _excluded_vars, _nodeps, _method_vars, _computed_vars, _vars, _defaults, and _dynamic.

Classes may perform additional processing if required by defining the classmethod() hooks _pre_hook_process_vars(cls) and _post_hook_process_vars(cls, results). The latter may modify the dictionary results.

Must be called in a subclass of StateVars.

Parameters :

copy : function, optional

Default copier. This is used to make a copy of the passed parameter when initializing so that (typically) mutable objects are not shared among instances.

archive_check : bool

If True, then the default values are checked to ensure that they are archivable.

with_docs : {None, interface, obj, list of obj}

Parse these objects for documentation. If no documentation is provided in the current class, then these will be searched for documentation and it used if found.

Returns :

results : dict

New class dictionary including ‘__doc__’

wrap__init__ : function

This function should be used to wrap the original __init__() method to perform the assignments etc.

Examples

>>> class A(StateVars):
...     "A simple class"
...     _state_vars = [('a', 1, 'Param a'),
...                    ('b', 2, 'Param b'),
...                    ('c', Computed, 'A computed parameter'),
...                    ('c1', Computed(save=True),
...                        'A saved computed parameter'),
...                    ('e', Excluded, 'An excluded parameter'),
...                    ('e1', Excluded(3), 'Excluded with default'),
...                    ('f', Excluded)]
...     process_vars()
>>> print A.__doc__
A simple class

Usage: A(a=1, b=2)



Attributes
----------

**State Variables:**
    a: Param a
    b: Param b

**Computed Variables:**
    c: A computed parameter
    c1: A saved computed parameter

**Excluded Variables:**
    e: An excluded parameter
    e1: Excluded with default
    f: <no description>

mmf.objects.class_NamedArray(array_, names=None)

Return a class that is a subclass of array_.__class__ but that allows the elements to be accessed through a named member syntax.

For example, if array_ = array([1.0, 2.0]) and names = array([‘x’, ‘y’]) then c[0] == c.x, and c[1] == c.y

The preferred usage is to define a class which inherits from the returned class. The arguments will then defined the names and the default values for objects instantiated from this class. The advantage of proceeding this way is that the name of the class will be well defined and usefull rather than the generic “NamedArray”.

Examples

>>> from numpy import array
>>> a = array([0.0, 0.0, 0.0])
>>> n = array(['x', 'y', 'z'])
>>> class CoordArray(class_NamedArray(a, n)): 
...     pass
>>> c0 = CoordArray()
>>> c0
CoordArray([ 0.,  0.,  0.])

You can initialize the array from a sequence, and use keyword arguments to set specific values. If you do not initialize, then the default values are used. The initialization array need not be complete: it just initializes the first n values.

>>> c = CoordArray([1, 3], y=2)
>>> c
CoordArray([ 1.,  2.,  0.])
>>> (c.x, c.y)
(CoordArray([ 1.]), CoordArray([ 2.]))
>>> c.x = 3; c.y = 4
>>> print c
CoordArray(y=4.0, x=3.0, z=0.0)

Note that one may have repeated labels:

>>> a = array([[0, 0], [0, 0]])
>>> n = array([['x', 'y'], ['y', 'z']])
>>> class MatArray(class_NamedArray(a, n)): 
...     pass
>>> c = MatArray(x=1, y=2, z=3)
>>> c
MatArray([[1, 2],
       [2, 3]])
>>> c.y
MatArray([2, 2])
>>> c.y = [4, 5]
>>> c
MatArray([[1, 4],
       [5, 3]])

>>> a = array([0, 0, 0])
>>> n = array(['x', 'y', 'z'])
>>> class MatArray(class_NamedArray(a, n)): 
...     pass
>>> c = MatArray(vals=[1, 2, 3])
>>> c
MatArray([1, 2, 3])
>>> c.t = 6
>>> c.t
6

mmf.objects.recview(a, names, formats=None, dtype=None)

Return a view of a as a recarray with defined field names.

>>> from numpy import array
>>> a = array([[1, 2, 3], [4, 5, 6]])
>>> r = recview(a, ['x', 'y', 'z'])
>>> print r
[[(1, 2, 3)]
 [(4, 5, 6)]]
>>> r.x = 0
>>> a
array([[0, 2, 3],
       [0, 5, 6]])