This Page

mmf.utils.mmf_theano

Polynomial([d]) Theano wrapper for polynomials.
Elemwise(scalar_op[, inplace_pattern, name, ...]) Wrapper for theano.tensor.Elemwise that overloads
UnaryScalarOp([d, name, _proxy]) Theano wrapper for functions of a single variable that can compute their own derivative via a flag d.
InverseUnaryScalarOp([name, _proxy]) Class skeleton to represent the inverse of an operator.
ScalarOpFromExpr(inputs, outputs[, d, name, ...]) This class allows you to construct a theano.scalar.ScalarOp
UnaryScalarOpFromExpr(x, f_x[, d, name, fn, ...]) Unary version of ScalarOpFromExpr. Takes a single variable x

Inheritance diagram for mmf.utils.mmf_theano:

Inheritance diagram of mmf.utils.mmf_theano

Tools for working with theano.

The main goal is to enable as seamless integration of custom functions as possible. In particular, addressing the following issues:

  1. Some functions may be purely numeric due to a complicated or adaptive algorithm. These functions need to be wrapped by theano.Op or theano.scalar.ScalarOp so that they can occupy a place in the tree. Once a function tree is generated, we can apply theano.tensor.Elemwise to allow vectorization, and then use theano.function() to compile this
  2. Compiling functions can be slow and should only be done when needed (maybe on import or at most once per session). An issue is how to use automatic differentiation of member functions that depend on parameters. Just passing symbolics through the function often works, but generates a tree that has the current values hard-coded: The function will need to be recompiled whenever the parameters change.
class mmf.utils.mmf_theano.Polynomial(d=0)[source]

Bases: theano.gof.op.Op

Theano wrapper for polynomials.

Examples

>>> import theano.tensor as T
>>> x = T.scalar('x')
>>> c = T.vector('c')
>>> p = Polynomial()(c, x)
>>> dp = T.grad(p, x)
>>> ddp = T.grad(dp, x)
>>> f = theano.function([c, x], [p, dp, ddp], mode='FAST_COMPILE')
>>> _c = [1.0,2.0,-3.0,4.0]
>>> f(_c, 2.0)
[25.0, array(38.0), array(42.0)]

We can also vectorize the polynomial over x. Note the use of sum() when constructing the derivatives. See https://groups.google.com/d/topic/theano-users/81hG-LzvxOY/discussion >>> import theano.tensor as T >>> x = T.matrix(‘x’) >>> c = T.vector(‘c’) >>> p = Polynomial()(c, x) >>> dp = T.grad(p.sum(), x) >>> ddp = T.grad(dp.sum(), x) >>> f = theano.function([c, x], [p, dp, ddp], mode=’FAST_COMPILE’) >>> _c = [1.0,2.0,-3.0,4.0] >>> _x = [[1.0,2.0],[3.0,4.0]] >>> f(_c, _x) [array([[ 4., 25.],

[ 88., 217.]]),
array([[ 8., 38.],
[ 92., 170.]]),
array([[ 18., 42.],
[ 66., 90.]])]

Attributes

default_output

Methods

__call__(*inputs, **kwargs) Optional: Return some or all output[s] of make_node.
c_code(node, name, inputs, outputs, sub) Required: Return the C implementation of an Op.
c_code_cache_version() Return a tuple of integers indicating the version of this Op.
c_code_cache_version_apply(node) Return a tuple of integers indicating the version of this Op.
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers() Optional: Return a list of header files required by code returned by
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code() Optional: Return utility code for use by a Variable or Op to be
c_support_code_apply(node, name) Optional: Return utility code for use by an Op that will be inserted at global
grad(inputs, output_gradients) We do not yet support differentiation wrt c though this should be
make_node(c, x)
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
perform(node, inputs, output_storage)
__init__(d=0)[source]
grad(inputs, output_gradients)[source]

We do not yet support differentiation wrt c though this should be easy (except for broadcasting issues).

make_node(c, x)[source]
perform(node, inputs, output_storage)[source]
class mmf.utils.mmf_theano.Elemwise(scalar_op, inplace_pattern={}, name=None, nfunc_spec=None)[source]

Bases: theano.tensor.elemwise.Elemwise

Wrapper for theano.tensor.Elemwise that overloads __call__() to directly call the implementation for numerical arguments.

Attributes

default_output

Methods

R_op(inputs, eval_points)
__call__(*v, **kw)
c_code(node, nodename, inames, onames, sub)
c_code_cache_version() Return a tuple of integers indicating the version of this Op.
c_code_cache_version_apply(node)
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers()
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code()
c_support_code_apply(node, nodename)
grad(inputs, ograds)
infer_shape(node, i_shapes)
make_node(*inputs) If the inputs have different number of dimensions, their shape
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
perform(node, inputs, output_storage)

Usage: Elemwise(scalar_op, inplace_pattern = {})

  • scalar_op: an instance of a subclass of scalar.ScalarOp which works uniquely on

    scalars

  • inplace_pattern: a dictionary that maps the index of an output to the

    index of an input so the output is calculated inplace using the input’s storage. (Just like destroymap, but without the lists.)

  • nfunc_spec: either None or a tuple of three elements, (nfunc_name, nin, nout) such

    that getattr(numpy, nfunc_name) implements this operation, takes nin inputs and abs(nout) outputs (nout < 0 if the numpy function does not provide the option of providing a numpy array to store the results in). Note that nin cannot always be inferred from the scalar op’s own nin field because that value is sometimes 0 (meaning a variable number of inputs), whereas the numpy function may not have varargs. NOTE: as of now, the sign of the nout field is ignored (some work needs to be done to resize the destinations when needed).

Attributes

default_output

Methods

R_op(inputs, eval_points)
__call__(*v, **kw)
c_code(node, nodename, inames, onames, sub)
c_code_cache_version() Return a tuple of integers indicating the version of this Op.
c_code_cache_version_apply(node)
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers()
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code()
c_support_code_apply(node, nodename)
grad(inputs, ograds)
infer_shape(node, i_shapes)
make_node(*inputs) If the inputs have different number of dimensions, their shape
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
perform(node, inputs, output_storage)
__init__(scalar_op, inplace_pattern={}, name=None, nfunc_spec=None)

Usage: Elemwise(scalar_op, inplace_pattern = {})

  • scalar_op: an instance of a subclass of scalar.ScalarOp which works uniquely on

    scalars

  • inplace_pattern: a dictionary that maps the index of an output to the

    index of an input so the output is calculated inplace using the input’s storage. (Just like destroymap, but without the lists.)

  • nfunc_spec: either None or a tuple of three elements, (nfunc_name, nin, nout) such

    that getattr(numpy, nfunc_name) implements this operation, takes nin inputs and abs(nout) outputs (nout < 0 if the numpy function does not provide the option of providing a numpy array to store the results in). Note that nin cannot always be inferred from the scalar op’s own nin field because that value is sometimes 0 (meaning a variable number of inputs), whereas the numpy function may not have varargs. NOTE: as of now, the sign of the nout field is ignored (some work needs to be done to resize the destinations when needed).

class mmf.utils.mmf_theano.UnaryScalarOp(d=0, name=None, _proxy=None)[source]

Bases: theano.scalar.basic.UnaryScalarOp

Theano wrapper for functions of a single variable that can compute their own derivative via a flag d. To use this, simply subclass and define impl(). Several things happen automatically:

  1. If not provided, the name will be set to the lowercase version of the class name with primes or ‘^{(d)}’ used to denote derivatives.
  2. The output_types_preference is set to theano.scalar.upgrade_to_float_no_complex (can override by defining as a class variable.)
  3. __call__() is overloaded to call impl() if none of the inputs is a theano variable. This will use variables names as listed in the arguments of impl() as the theano variables, and will either use the class variables of the same name or will use _variable_type to create the variables.
  4. Provides a convenience method get_function() that returns the compiled function, inspecting impl() to determine the arguments and using _variable_type to define the types.

Attributes

default_output

Methods

__call__(*v, **kw)
c_code(node, name, inputs, outputs, sub) Required: Return the C implementation of an Op.
c_code_cache_version()
c_code_cache_version_apply(node) Return a tuple of integers indicating the version of this Op.
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers() Optional: Return a list of header files required by code returned by
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code() Optional: Return utility code for use by a Variable or Op to be
c_support_code_apply(node, name) Optional: Return utility code for use by an Op that will be inserted at global
get_function([vectorize]) Return the op as a compiled function.
grad(inputs, output_gradients)
impl(*v, **kw) Required.
make_node(*inputs)
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
output_types(types)
perform(node, inputs, output_storage)
__init__(d=0, name=None, _proxy=None)[source]
elemwise[source]

Return as a vectorized op.

get_function(vectorize=True)[source]

Return the op as a compiled function. Inspects impl() to determine the names of the arguments. This will of course fail if _proxy is used.

grad(inputs, output_gradients)[source]
impl(*v, **kw)[source]

Required. Compute and return the `self.d`th derivative numerically.

class mmf.utils.mmf_theano.InverseUnaryScalarOp(name=None, _proxy=None)[source]

Bases: mmf.utils.mmf_theano.UnaryScalarOp

Class skeleton to represent the inverse of an operator. You must provide the implementation (presumably using a root-finder of some sort. Presently you need to provide an operator (not applied to its arguments) for both f and its derivative df. This could be relaxed if we figure out how to “unapply” an operation.

Notation, y=f(x), this function thus has argument y.

Attributes

default_output

Methods

__call__(*v, **kw)
c_code(node, name, inputs, outputs, sub) Required: Return the C implementation of an Op.
c_code_cache_version()
c_code_cache_version_apply(node) Return a tuple of integers indicating the version of this Op.
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers() Optional: Return a list of header files required by code returned by
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code() Optional: Return utility code for use by a Variable or Op to be
c_support_code_apply(node, name) Optional: Return utility code for use by an Op that will be inserted at global
get_function([vectorize]) Return the op as a compiled function.
grad((y,), (gz,)) This is the key: we use self to compute x then return 1/df.
impl(*v, **kw) Required.
make_node(*inputs)
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
output_types(types)
perform(node, inputs, output_storage)
__init__(name=None, _proxy=None)[source]
df = NotImplemented
f = NotImplemented
grad((y, ), (gz, ))[source]

This is the key: we use self to compute x then return 1/df.

impl(*v, **kw)[source]

Required. Compute and return the inverse function numerically.

class mmf.utils.mmf_theano.ScalarOpFromExpr(inputs, outputs, d=None, name=None, fn=None, mode='FAST_COMPILE')[source]

Bases: theano.scalar.basic.ScalarOp

This class allows you to construct a theano.scalar.ScalarOp from a tensor expression. This allows for automatic derivative calculations.

Attributes

default_output

Methods

__call__(*inputs, **kwargs) Optional: Return some or all output[s] of make_node.
c_code(node, name, inputs, outputs, sub) Required: Return the C implementation of an Op.
c_code_cache_version()
c_code_cache_version_apply(node) Return a tuple of integers indicating the version of this Op.
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers() Optional: Return a list of header files required by code returned by
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code() Optional: Return utility code for use by a Variable or Op to be
c_support_code_apply(node, name) Optional: Return utility code for use by an Op that will be inserted at global
grad(inputs, grads)
impl(inputs)
make_node(*inputs)
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
output_types()
perform(node, inputs, output_storage)
__init__(inputs, outputs, d=None, name=None, fn=None, mode='FAST_COMPILE')[source]
grad(inputs, grads)[source]
impl(inputs)[source]
output_types()[source]
class mmf.utils.mmf_theano.UnaryScalarOpFromExpr(x, f_x, d=0, name=None, fn=None, mode='FAST_COMPILE')[source]

Bases: mmf.utils.mmf_theano.ScalarOpFromExpr

Unary version of ScalarOpFromExpr. Takes a single variable x and a single expression f_x as inputs instead of lists. The derivatives are also specified as an integer d now.

Attributes

default_output

Methods

__call__(*inputs, **kwargs) Optional: Return some or all output[s] of make_node.
c_code(node, name, inputs, outputs, sub) Required: Return the C implementation of an Op.
c_code_cache_version()
c_code_cache_version_apply(node) Return a tuple of integers indicating the version of this Op.
c_code_cleanup(node, name, inputs, outputs, sub) Optional: Return C code to run after c_code, whether it failed or not.
c_compile_args() Optional: Return a list of compile args recommended to compile the
c_header_dirs() Optional: Return a list of header search paths required by code returned by
c_headers() Optional: Return a list of header files required by code returned by
c_lib_dirs() Optional: Return a list of library search paths required by code returned by
c_libraries() Optional: Return a list of libraries required by code returned by
c_no_compile_args() Optional: Return a list of incompatible gcc compiler arguments.
c_support_code() Optional: Return utility code for use by a Variable or Op to be
c_support_code_apply(node, name) Optional: Return utility code for use by an Op that will be inserted at global
grad((x,), (gf,))
impl(x)
make_node(*inputs)
make_thunk(node, storage_map, compute_map, ...)
param node:something previously returned by self.make_node
output_types(_dtypes)
perform(node, inputs, output_storage)
__init__(x, f_x, d=0, name=None, fn=None, mode='FAST_COMPILE')[source]
grad((x, ), (gf, ))[source]
impl(x)[source]
nin = 1
nout = 1
output_types(_dtypes)[source]