Configuration - part 4

We've seen how to dynamically load class with the ConfigurationManager in my previous posting. The configuration manager interrogates the loaded class to find any further configuration requirements it may have. These additional parameters are added to the configuration requirements for the whole application.

There is a potential conflict though. For example, let's say we've got an application that copies data from a source to a destination. Further, we've a hierarchy of classes that define potential sources and destinations.

We could specify a PostgreSQL source and an HBase destination. A database wants a hostname, port number, username and password. HBase wants only the hostname and port number. If both classes define the configuration parameter 'hostname', we've got a conflict. A valid hostname for the database isn't likely to be the valid name for HBase.

Namespaces can resolve this issue by prefixing and isolating any symbols within them.

    top_level = cm.Namespace()
    top_level.source = cm.Namespace(doc='the input source')
    top.level.source.option('storageClass',
                 doc='the classname for the source',
                 default='socorro.storage.crashstorage.DatabaseCrashStorage',
                 from_string_converer=cm.class_converter)
    top_level.destination = cm.Namespace(doc='the output destination')
    top_level.destination.option('storageClass',
                 doc='the classname for the destination',
                 default='socorro.storage.crashstorage.HBaseCrashStorage',
                 from_string_converter=cm.class_converter)

When finally using these definitions in an application, you'll find that the requirements of the two loaded classes are prefixed with 'source' and 'destination':

    print config.source.hostname
    print config.source.port
    print config.destination.hostname
    print config.destination.port

The conflicts are resolved.

How does this work for the command line? here's how you'd specify HBase as both a source and a destination:

    python sample.py   
        --source.classname=socorro.storage.crashstorage.HBaseCrashStorage
        --source.hostname=hbase1 --source.port=9090 
        --destination.classname=socorro.storage.crashstorage.HBaseCrashStorage
        --destination.hostname=hbase2
        --destination.port=9090

This has a direct analog in the flat conf file and the ini files. in the conf files, we've got the same dot separated structure:

    source.storageClass=socorro.storage.crashstorage.HBaseCrashStorage
    source.hostname=hbase1
    destination.storageClass=socorro.storage.crashstorage.HBaseCrashStorage
    destination.hostname=hbase2

In the ini files, the namespaces translate into the sections in the ini file.

    [source]
    storageClass=socorro.storage.crashstorage.HBaseCrashStorage
    hostname=hbase1
    [destination]
    storageClass=socorro.storage.crashstorage.HBaseCrashStorage
    hostname=hbase2

As noted in part 1 of this series. you can get the whatever app employs the ConfigurationManager to write these config files for you with the --_write=ini or --_write=conf command line options.

Next, we'll see how to make the entire guts of an application configurable.