Summary

These two features will allow developers to test changes to the gateway without needing a full network setup with devices and XBees.

After talking to the current firmware team (who is responsible for the gateway at the moment), I've finally managed to get around to some improvements for testing - two different modes that don't require a full weatherbox to be operational for gateway development!

(env) kluong@kserver:~/control_tower/gateway/src$ python gateway_server.py fake
Setup
Setup XBee Device
Starting main Gateway Loop

(env) kluong@kserver:~/control_tower/gateway/src$ python gateway_server.py bridged /dev/ttyACM0
Setup
Setup XBee Device
Starting main Gateway Loop

Fake mode will work on scelserver without any additional modifications, so feel free to give it a try when you get a chance.

These two modes allow for testing the gateway without having a working weatherbox. When used, these are how the modes work:

Mode	Description
Fake Mode	This mode does not require any physical devices at all; it simulates a weatherbox network by sending packets from multiple types of weatherboxes in an infinite loop.
Bridged Mode	This mode allows you to hook up a physical device that emits TX frames onto uart lines. This could mean a full weatherbox, or just a microcontroller running a stubbed version of the normal firmware.

This is what the normal pipeline looks like:

weatherbox:gateway:normal_mode.png

The microcontroller on the weatherbox side sends a transmit request to the remote xbee:

The coordinator xbee (attached too the gateway), then sends a receive packet frame through uart:

So, if this is how things normally work, we'd need some hardware to test the gateway to see that it's decoding packets correctly. We'd need:

A weatherbox
2 XBees
A usb serial adapter to connect to the XBee on the hardware side.

This is problematic for a couple of reasons:

Setup is complicated and cumbersome
Setup is expensive

I realized that if I could have some script running that could translate transmit request frames to receive frames on the other side we could connect a weatherbox (or microcontroller) directly to a gateway to test it:

weatherbox:gateway:bridged_mode.png

Any firmware that emits transmit frames onto a UART line connected to a computer will work; in my case, I used a modified version of the ga_stub_hb firmware build loaded onto a stock arduino to accomplish this. The only modification I made was to have the xbee driver write to the hardware serial lines instead of the software serial ports. This allows you to just use the arduino, instead of having to have a separate serial usb converter.

Fake mode takes bridged mode one step further and takes all hardware away completely:

weatherbox:gateway:fake_mode.png

The packets that are sent are hardcoded into the FakeNetwork class:

     self.packets = {}
     self.packets['heartbeat'] = "\x7e\x00\x16\x90\x00\x7d\x33\xa2\x00\x40\xe6\x4b\x5e\x03\xfd\x01\x00\x00\xff\xff\xf0\xfa\x23\x00\x2b\x02\xb2"
     self.packets['apple'] = "\x7e\x00\x22\x90\x00\x7d\x33\xa2\x00\x40\x9f\x27\xa7\x29\x6c\x01\x01\x00\xff\xff\x80\x6f\x69\x3d\x06\x0f\x71\x7d\x33\x5a\x8a\x01\x00\x76\x01\x22\x00\x6e\x09\x55"
     self.packets['cranberry'] = "\x7e\x00\x22\x90\x00\x7d\x33\xa2\x00\x41\x25\xe5\x88\x0c\x83\x01\x02\x00\xff\xff\x7c\xf3\x05\x00\xba\x0f\x5c\x08\x05\x00\x20\x73\x3b\x00\xdd\x8b\x01\x00\x7a"
     self.packets['dragonfruit'] = "\x7e\x00\x24\x90\x00\x7d\x33\xa2\x00\x40\xe6\x72\x7d\x5e\x30\x18\x01\x03\x00\xff\xff\x30\xc8\x07\x00\x6b\x0d\xf4\x00\x06\x00\x00\x00\xb6\x72\x37\x00\xfe\x8b\x01\x00\x00"
     self.packets['snapdragon'] = "\x7e\x00\x22\x90\x00\x7d\x33\xa2\x00\x40\xa3\x53\x7d\x5e\x20\x9c\x01\x04\x00\xff\xff\x12\xe4\x49\x00\xca\x0d\x44\x0c\x2c\x31\x01\x00\x2f\x01\x34\x00\x64\x00\xbb"
     self.packets['badPacket'] = "\x7e\x00\x16\x90\x00\x7d\x33\xa2\x00\x40\xe6\x4b\x5e\x03\xfd\x01\x00\x59\x90\x95\x95\x94\x88\x48\x48\x49\xe2"

These packets were taken a while back by grabbing receive frames from an xbee.

It's important to have a testing process that easy and can be done without the production environment, otherwise you'll always be trying to test in production!

Item	Description
Automated testing: t	This is important, because as the software gets bigger it gets harder and harder to test all of the cases manually by running the whole program.
Unit testing
End-to-end testing
Packaging story needs to be better; hard to run application with virtualenv
Fake mode with C code - this might be useful for creating new schemas
Deployment needs to be better * systemd * staging environment?
Command line args could use some cleanup
Is there a way to de-couple the python xbee library from the underlying transport mechanism
We could possibly use a pty to “subscribe” via some mechanism for debugging and testing purposes
REST api to get status?
Adding a new schema to the gateway
Logs for the gateway could be more useful
Migrate from python 2 to python 3	Python 2 is going away