Things Gateway - Bonding Philips HUE Lights

The Things Gateway has a rules system that allows an action taken on one thing to cause an effect on another thing.  In version 0.4, there is a one to one relationship: a change to one thing can induce a change to one other thing.  In the upcoming 0.5 version, the relationship can be one to many: a change in the state of one thing can cascade to a change in many things.  However, there is still no way to propagate a state, like color, from one thing to another.

The Web Thing API opens the opportunity to use any Web capable language to create arbitrarily complex rules.  Armed with an authorization key, any program you create can interact with the Things Gateway to manipulate the things in your home. 

In today's project, I'm going use the Web Thing API to bond a set of four Philips HUE lights together so they act in unison. A change to the on/off state or color of any one bulb from the Things Gateway user interface will immediately be echoed by the other bulbs.

Check out the video to see it in action.


I'll start with how to use my code before I talk about how it works.  Here are the requirements if you'd like to try this project for yourself:

Item What's it for? Where I got it
A Raspberry Pi running the Things Gateway with the associated hardware from Part 2 of this series. This is the base platform that we'll be adding onto From Part 2 of this series
DIGI XStick This allows the Raspberry Pi to talk the ZigBee protocol - there are several models, make sure you get the XU-Z11 model. The only place that I could find this was Mouser Electronics
Several Philips Hue White & Color Ambiance bulbs paired with the Gateway This will be the Web Things that get bonded together to act in unison.  Set up one with a HUE Bridge with instructions from Part 4 of this series or independently from Part 5 of this series. Home Depot
a computer with Python 3.6 My bonded_things.py code was written with Python 3.6.  The  RPi that runs the Things Gateway has only 3.5.  To run my code, you'll need to either install 3.6 on the RPi or run the bonded_things program on another machine. My workstation has Python 3.6 by default


Step 1: Install the software modules required to run bonded_things.py:
        
    $ sudo pip3 install configman
    $ sudo pip3 install webthing
    $ git clone https://github.com/twobraids/pywot.git
    $ cd pywot
    $ export PYTHONPATH=$PYTHONPATH:$PWD
    $ cd demo
    $ cp bonded_things_sample.ini bonded_things.ini

Step 2: You're going to need to have the thing_id for each of the HUE lights that you want to bond together.  The fastest way to get those from the user interface is to click on the "splat" on each light in turn.  The thing_id appears in the URL bar as the last string of characters.  In the case below, it is "zb-001788010311383e"

Edit the bonded_things.ini file to add your comma delimited list of thing_ids in the manner shown here:

        
# a list of thing ids to bond together
list_of_thing_ids=zb-001788010311382c, zb-001780311383e, zb-001780360df9c, zb-0017803415d70

# the api key to access the Things Gateway
things_gateway_auth_key=


You are also going to need an authorization key to communicate with the Things Gateway

The Things Gateway UI will generate an authorization key for you by going to
≡ ⇒ Settings ⇒ Authorizations ⇒ Create New Local Authorization ⇒ Allow


You can even create authorizations that allow access only to specific things within your gateway.   Once you've pressed "Allow", the next screen gives you the Authorization Token as well as examples of its use in various languages:


Copy your authorization key into the appropriate place in the bonded_things.ini file.

Step 3: Run the bonded_things like this:
        
    $ ./bonded_things.py --admin.conf=bonded_things.ini


All of the lights corresponding to the thing_ids that you put in the configuration file will now work in unison.

How it works:

There are two ways for a program to communicate with Web Things through the Things Gateway: via a RESTful API and a Web Sockets API.  I'm going to use both methods to write the bonded_things controller.

The RESTful API allows an external program to query and manipulate the state of Web Things.  It doesn't, however, allow the external program to be notified of state changes.  A program restricted to using the RESTful API alone is relegated to caching state and polling to detect changes.

The Web Socket API eliminates the need for polling. Think of it as subscribing to changes of state of a Web Thing.  The external program only needs to listen on a socket for incoming data about state changes when they happen.   There is a one to one relationship between a Web socket and a Web Thing.  The external program must open a Web socket to each Web Thing of interest.  If you want  to monitor the state of one hundred Web Things, you will need one hundred Web sockets.

The bonded_things.py program is an exercise in asynchronous programming. It begins by taking a list of Web Thing ids and starting an asynchronous function (coroutine) for each one.
async def bond_things_together(config):
    for a_thing_id in config.list_of_thing_ids:
        asyncio.ensure_future(
            monitor_and_propagate_state(config, a_thing_id)
        )
(see this code in situ in the bonded_things.py file in the pywot demo directory)

Each coroutine opens a Web socket to its assigned Web Thing and starts listening. When a user of the Things Gateway UI turns a Web Thing on or off, or, perhaps, changes a color, a message is sent via the open Web socket.  The coroutine's response is to take the changed property from the message and then use the RESTful API to propagate the same changes to the other Web Things in the bonded group.
    async with websockets.connect(
        'ws://gateway.local/things/{}?jwt={}'.format(
            thing_id,
            config.things_gateway_auth_key
        ),
    ) as websocket:
        async for a_message_txt in websocket:
            a_message = json.loads(a_message_txt)
            if a_message['messageType'] == 'propertyStatus':
                for a_property in a_message["data"].keys():
                    a_value = a_message["data"][a_property]
                    change_property_for_all_things(config, thing_id, a_property, a_value)
But wait, when each one of those other things has its state changed by the RESTful API, it too sends a changed state message on the Web sockets being run by other coroutines.  That would cause each one of those coroutines to use the RESTful API to change the others, just like the first one was doing.  We don't want a runaway feed back loop.  We can avoid the problem by adding a counted semaphore that ensures that only the correct number of state changes get propagated.

    global suppress_state_change
    async with websockets.connect(
        'ws://gateway.local/things/{}?jwt={}'.format(
            thing_id,
            config.things_gateway_auth_key
        ),
    ) as websocket:
        async for a_message_txt in websocket:
            a_message = json.loads(a_message_txt)
            if a_message['messageType'] == 'propertyStatus':
                if suppress_state_change:
                    suppress_state_change -= 1
                    continue
                for a_property in a_message["data"].keys():
                    a_value = a_message["data"][a_property]
                    suppress_state_change = number_of_bonded_things - 1
                    change_property_for_all_things(config, thing_id, a_property, a_value)
(see this code in situ in the bonded_things.py file in the pywot demo directory)

Imagine four instances of this method running cooperatively.  The variable suppress_state_change behaves like a counting semaphore.  When one instance of the coroutine acts on a message, the semaphore is set to the number of things in the bonded group minus one.  Each time a thing's coroutine receives a message, if suppress_state_change is not zero, it decrements the semaphore and throws the message away.  This stops the positive feedback loop.

That's really all there is to it.

This method of using Web sockets to watch for changes in the state of Web Things could be the basis for a scene control system: a set of lights and/or plugs  working together as if they were one device.  Today's project is a first step, a proof of concept, that reveals the features of the Web Thing API.

In the next blog post, I'll use these same features to create an externally implemented scene control system that learns by example.  Here's a preview: