Deep AI design for Home Automation

After doing some lovely reading of Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow, I now have the tools to apply deep learning techniques to improve our Home control predictions. Here lists a concepts I want to try and experiment with:

Deep Neural Networks

My initial intuition is that this will not be miles ahead of a base Decision Tree or Logistic Regression model but will still give an useful benchmark.

The basis is still to use the base neural networks for classification of activator states based on sensor states. This will be applying it to the same data structure as we have used before, with the caveat

input = sensor_1_state, sensor_2_state, sensor_3_state…

output = activator_state_1, activator_state_2, activator_state_3…

Having activator states as input could help in scenarios where the activities are interconnected (turn off lights in living room when bedroom lights are on). A caveat of this to stop loops is that output changes here do not trigger a prediction. In this case:

input = activator_state_1,activator_state_2, … sensor_1_state, sensor_2_state, …

output = activator_state_1, activator_state_2, activator_state_3…


The thinking behind using this is that past sequences of sensor states will provide useful information that is missed. IF I came from the living room then don’t turn on the corridor lights but IF I came from outside then do turn on the corridor lights. However, note that the length of steps between each state change capture might be important here hence it will be an engineered feature.

Data structure:

input: device1 , state , t=1 device2 , state , t=2 device1 , state , t=3

output: device x, state x


input: activator_state_1, activator_state_2, … sensor_1_state, sensor_2_state, … t=1 activator_state_1, activator_state_2, … sensor_1_state, sensor_2_state, … t=2 activator_state_1, activator_state_2, … sensor_1_state, sensor_2_state, … t=3

output: activator_state_1, activator_state_2, activator_state_3…

I plan to try the following architectures:

  • Gated Recurrent Unit
    • Good approach to the problem:
    • Shorter term memory
  • Conv1D
    • Longer term memory

Proposed Architecture


Optimizing techniques

  • Dropouts
    • Definite must try to see if training speed will be better.
    • Applied to input and output
  • L1 & L2 Regularization

Feature engineering

time_to_change - This is the current last_updated_time minus the previous last_updated_time. This should help the model segregate between useful state changes or not useful. If time_to_change > 1 min then its not really a sequence.

duplicated - There could be duplicated states due to periodic capture them. These should be weighted less as our focus is the change of actuator states.

Future considerations

Online Learning

It is key to answer - How do our behaviors change and at what frequency? There is a balance between overfitting to chaotic changes in our daily schedules. Small fluctuations should not be weighted too much.

Re-running the training process day-to-day on the personal dataset may be a suitable technique to learn day to day behavioral changes (i.e. start using the new side desk lamp). This can also be combined with training on the ‘error’ cases where we explicitly reverse an action.

Transfer learning

One future idea is to train a generic model on a large dataset with multiple users. This model will answer What type of person are you and what automations can help you increase your comfort?