user:kluong:improvement_-_simplified_data_pipeline

This is an old revision of the document!


Improvement - simplified data pipeline

Note: this page is in draft.

There's a bit of complexity around how handle sensor data. Here are two things:

  • Every generation has it's own packet format
  • There's a different table for each generation

At the time, this was an slightly arbitrary decision that I personally made, thinking that this would make it easier to manage multiple generations of sensor boxes. In reality, I think it makes things way more complicated than they need to. Here's what I think should happen:

  • Firmware packets are standard across all hardware types
  • The database is consolidated to one table that stores all sensor data

More thoughts here

* https://wiki.scel-hawaii.org/doku.php?id=user:kluong:thinking_about_a_generic_schema

  • user/kluong/improvement_-_simplified_data_pipeline.1616800089.txt.gz
  • Last modified: 2021/09/19 21:59
  • (external edit)