Example

Suppose a program needs to run a series of tasks, each with separate, but sometimes similar, parameters. Additionally, suppose the user is only interested in running a subset of the tasks she has parameters written for. This can be expressed as:

# This is a list of the tasks to run, each linked to their
# definitions below.  This list will be returned to the user as
# a list of subtrees, themselves PyOptionTrees.

Tasks = [jog, eatcereal, eattoast]

# The { } defines a subtree.  The subtree is itself an
# PyOptionTree.
jog = {
  name = "Go Jogging"
  action = "jog"
  minutes = 30
  location = "park"
  # etc.
}

# This subtree won't show up in the above list
ridebike = {
  name = "Ride the bike"
  action = "ride bike"
  minutes = 90
  location = "trail by house"
  # etc.
}

eatcereal = {
  name = "Eat Breakfast"
  action = "eat"
  food = "cereal"
  location = "kitchen"
  # etc.
}

Now suppose all the parameters from 'eattoast' are the same as those of 'eatceral' except the value of the 'food' parameter. We can copy the whole subtree and overwrite that parameter:

eattoast = copy(eatcereal)
eattoast/food = "toast"

In our program, we could load and use the above with:

def runTests(opttreefile):
    o = PyOptionTree(opttreefile)
    for t in o.Tasks:      # According to the above definition,
        runTest(t)         # t is a subtree (branch).

def runTest(ot):
    print 'Current Action: ', ot.name
    # Decide what do based on ot.action...
    # Do the action...

Now that we've looked at a quick example, let's move on to the good stuff.

Syntax

Every option file is simply a list of statements of the form <name> = <value>. A single space or and end-delimiting character is sufficient to separate most <value> definitions from the next statement; if this presents a problem, a ';' may be used.

The value can be a string, number, list, tuple, another name, a string of python code to evaluate, a tree node, or a function call. The standard syntax for each value type is described below.

Names can consist of any alphanumeric characters or underscores. Referencing names not in the local tree is done exactly like on *nix filesystem: local names are separated by'/', '..' moves up the tree, and starting with a '/' starts the referencing from the root node of the tree (be careful using this; the target changes between when the tree is read in as a subtree and when it is the base tree). Thus:

../tree1/opt1 = 5

moves up one level and sets the value of opt1 inside tree1 to 5. If trees do not exist, they are created (so the above command would create 'tree1' if it didn't exist.)

Assignments with the same name are always resolved to the last one specified. In the case of trees; subsequent trees of the same name are merged into previous trees; otherwise, when reading in a tag, any previously stored value with that name is overwritten.

ot.get('tree/subtree/element')
ot('tree/subtree/element')
ot.tree.subtree.element

Basic Value Types

Possible types of the <value> token:

Strings

Strings start with either " or ' and end with the same character (same as Python).

Numbers

Numbers can include digits, a decimal mark (period), a minus sign, and E notation without spaces (e.g. 1.45e-5 or 14.1E56).

Boolean

Boolean values are denoted by the keywords 'True' and 'False'.

None

'None' (no quotes in file) or ';' return the null value None.

Links

Links are simply the name of another node. They can be thought of like soft links in a Linux file system; when the option tree retrieves a parameter whose value is a link, it returns whatever value the link (or chain of links) points to. (Note that recursive links will result in an PyOptionTreeResolution exception at runtime.)

There are two issues to be aware of. The first is that, if the name of the assignment resolves to another tree (e.g. 'tree1/o1 = 1' or '../o1 = 1') links are always resolved starting from the location of the local assignment. For example, the following is correct:

o1 = 1
tree1/o2 = ../o1

but the following is not:

o1 = 1
tree1/o2 = o1

This is to avoid confusion with how nodes are handled, i.e. so the statements 'tree1/o2 = ../o1' and 'tree1 = {o2 = ../o1}' are identical.

The second issue is that links are only resolved when the user requests a value, thus in the following code:

o1 = 1
o2 = o1
o1 = 2

retrieving o2 will return 2. (See the copy() function for different behavior).

Lists

Lists, as in Python, start with '[' and end with ']' and contain a comma separated sequence of zero or more basic values. Nested lists are allowed.

Links involving lists may contain references to specific elements, or sublists of elements. The syntax is standard Python. For example, the following are all valid, and produce the expected behavior:

l = [1,2,3,4,5]
i1 = l[4]
i2 = l[-1]
l1 = l[1:2]
l2 = l[:-1]

Additionally, assignments modifying individual elements are allowed:

l = [1,2,3,4,5]
l[0] = 10

Here, l would return [10,2,3,4,5]

eval(...), @(...)

Any valid Python statement that returns a value can be embedded in the option tree file using '@' or 'eval' followed by parenthesis. Values of other option tree names and local functions can be included in the statement using '$' followed by parenthesis containing a link to the value. In the following code:

o1 = 1
o2 = @(1 + $(o1))

o1 will be 1 and o2 will be 2.

Evaluation is done when the value is retrieved, unless forced by now() (see below). Normally, the functions available are identical to those returned by the global() function; however, the user may supply a dictionary of additional functions and/or values when retrieving the value using the get() method.

Tuples

Tuples have the same syntax as lists, except they start with '(' and end with ')'.

Subtrees

Subtrees are denoted with a pair of curly braces '{' '}'. In between is a list of <name> = <value> statements. Thus in the following example:

tree1 = {
  o1 = 5
}

the user would access o1 as opttree.tree1.o1 or opttree.get("tree1/o1"), assuming the option tree is in the opttree variable.

Note that including another option tree file using the optfile() function and simply putting the contents of that file inside curly braces result in the same option tree.

Functions

Functions take a comma separated sequence of basic types and return a value. The syntax is the same as Python (and a number of other languages). An example of the optfile() function (mentioned above and described below) is:

tree_from_file = optfile("othertree.opt")

There are a number of built in functions for basic operations, and the user may supply a list of additional functions which may be part of the parameter input. See the documentation on the addUserFunctions() method for more information.

Functions are normally evaluated when the user asks for the parameter they are referenced to, which allows the arguments to be links to items defined later. copy(), file(), now() and reref() are evaluated during the parsing, so any links given as arguments must point to items already defined.

The built in functions are:

add(arg1, ...), cat(arg1, ...), sum(arg1, ...)

All three of these have identical behavior and add up the list of arguments using the + operator in Python. Specifically, they calculate ((...((arg1 + arg2) + arg3) + ...) + argn).

copy(arg1 [, arg2, ...])

copy() takes one or more arguments (usually links to other trees) and returns a copy of each. If one argument is given, copy() returns a value; if multiple arguments are given, it returns a list. The copying is done by building an identical tree, making copies of all lists, tuples, dictionaries, and subtrees. All relative links are modified so they point to the same elements they did originally (see reref() for alternate behavior).

copy() and reref() are executed within the parsing, so all the arguments to copy must already be defined.

copy() allows the user to modify elements in a duplicate tree without actually changing the value in the original tree. For instance, in the following example:

tree1={o1 = 1}
tree2=tree1
tree2/o1 = 10

both tree1/o1 and tree2/o1 will hold the number 10. Using copy():

tree1={o1 = 1}
tree2=copy(tree1)
tree2/o1 = 10

results in tree1/o1 = 1 and tree2/o1 = 10.

Note that copying one base tree and modifying specific elements gives inheritance-like behavior (as in the first example).

dict(arg1, ...)

dict() takes a list of tuples and turns them into a python dictionary. The list may either be a comma separated list, an actual list type, or a link to a list.

optfile(arg1, ...)

optfile() reads a set of option tree parameters from file arg1 and returns an option tree containing them. arg1 may be either a string (or a link that ultimately resolves to a string) specifying the file, or a tuple containing two strings, the first being the file name and the second being the source name (used in error reporting and the tree description).

If multiple arguments are given, optfile() treats them as if they were continuations of the first file. In other words, it loads all the parameters into the same option tree, with conflicts being resolved in favor of the last name specified.

now(arg1)

Forces evaluation of any contained functions (including those pointed to by links) at parse time instead of at retrieval time. arg1 can be any basic value.

range([start, ] end [, step])

range() produces a list of evenly spaced values. If all arguments are given, it returns an ordered list of all numbers of the form start + i*step for i = 0,1,2,... such that start <= n < end if step is positive and start >= n > end if step is negative 0. If ommitted, start defaults to 0 and step to 1. If all arguments are integers, all the elements in the list are integers, otherwise they are floating points. The following are examples:

range(5) = [0,1,2,3,4]
range(-1,4) = [-1,0,1,2,3]
range(1.5,0,-0.2) = [1.5,1.3, 1.1, 0.9,0.7,0.5,0.3,0.1]

rep(string1, ...):

rep() replaces all the occurences of links in string1 and any additional strings with the string representation of their retrieved values. Links are specified using ${<link>} and are referenced from the current location. Thus in the following example:

name = 'Hoyt'
tree1 = {
  author=rep('This example was written by ${../name}')
}

tree1/author would equal 'This example was written by Hoyt'.

reref(arg1 [, arg2, ...])

reref() is identical to copy() except that only links given as direct arguments are followed; all other relative links within the tree are left unmodified. As an illustration of why this might be useful, consider the following, which specifies two lists of cities:

list1 = {
  style1 = "Bold"
  style2 = "Normal"

  # here a list of (name, style) tuples
  items = [("Colorado", style1),
           ("Denver", style2)]
}

list2 = {
  style1 = "Underlined"
  style2 = "Tiny"
  items = reref(../List1/items)
}

In this example, list1/items would return:

[("Colorado","Bold"), ("Denver", "Normal")],

and list2/items would return:

[("Colorado","Underlined"),("Denver", "Tiny")]

Note that the 'style1' link in the first item list points to list1/style1', whereas the reref() in 'list2' copies 'list1/items' to 'list2/items' without modifying the hyperlinks in the original; thus they now point to 'list2/style*'.

outer_product(opttree, field1, field2,...)

outerProduct() generates a list of new option trees based on opttree but with any lists in field1, field2, ... (string names of lists in opttree) reduced to scalars. Each resulting option tree has exaclty one combination of the elements in these lists, with the full list of trees having every combination.

For example, in:

Foo = {
  a = [1, 2]
  b = [3, 4]
  c = 12
}

Bar = outerProduct(Foo, 'a', 'b', 'c')

Bar would contain a list of four option trees:

Bar[0] = {a = 1; b = 3; c = 12 }
Bar[1] = {a = 1; b = 4; c = 12 }
Bar[2] = {a = 2; b = 3; c = 12 }
Bar[3] = {a = 2; b = 4; c = 12 }

unpickle(arg1,...), unpickle_string(arg1,...)

unpickle() loads a python object from a pickle file and returns it. unpickle_string(arg1, ...) is identical except that it assumes the pickle is imbedded within the option file in the string arg1, etc. If multiple arguments are given, both functions return a list of objects.

On error, both functions throw a PyOptionTreeRetrievalError exception.

Errors

Errors are handled by exceptions. They fall into three categories, parse errors, retrieval errors, and evaluation errors. Parse errors occur when parsing input and will raise a PyOptionTreeParseError exception. Retrieval errors occur when the user asks for a non-existant value or a link within the tree can't be resolved; these raise a PyOptionTreeRetrievalError exception. Evaluation Errors occur when code inside an evaluation statement raised an exception; these throw a PyOptionTreeEvaluationError exception.

All three exceptions are derived from the base class PyOptionTreeException and return the error message when str() is used on an instance.

Other Issues

If you have any problems with the code, run into cases where it doesn't work, are confused by the documentation, or have a comment, please email me at the address above. This project is still very much in beta test mode.

License (MIT License)

Copyright (c) 2007 Hoyt Koepke

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.