tom mckenzie

GPT-3 Home Control with Node-RED

20th Apr 2023

What you’ll need

Overview

I have several lights and an air conditioner in Homebridge. This shortcut joins a prompt (with info about the devices and their capability) with the user’s request, and feeds it into a Node-RED endpoint (on my local network) which controls the devices itself.

This enables behaviour such as:

GPT also replies with a friendly response, such as:

It also supports a delay property, so GPT can activate devices, then set them “off” after a period of time.

The shortcut

Note the prompt includes information about the smart home, and defines the response shape (in Typescript). This is necessary because the Node-RED endpoint pipes the output from GPT straight into the Homebridge inputs, so it needs to define the correct values based on the capabilities of the devices (i.e. coloured lights receive HSL values, normal lights only receive Brightness, Air Conditioner receives temperature values)

Open in Shortcuts

Wiring up the specific device identifiers in the shortcut to the ones defined in Node-RED is the trickiest part, now that we have the right shapes for the capabilities of dimmable lights, temperature lights, colored lights, and air conditioners defined.

The Node-RED flow

flows.json

This flow listens on /control and simply splits the actions array into instructions, then sends the instruction to the right device.

You’ll have to update the shortcut to point to your Node-RED IP, and update the devices to be connected to your own devices.

Summary and improvements

This works surprisingly well, especially if you give it direct instructions! It knows the current time and temperature, so it will set intuitive values when it’s too warm or cold, and you can throw it a general vibe and it’ll pick up on what you mean. It even supports bisexual lighting out of the box.

GPT doesn’t seem to be pro-active about taking actions based on the time of day, doesn’t know what rooms or the configuration of your lights (my two “center” lights often get set together, but sometimes it’ll only turn on/off one of them).

It also doesn’t know about the current state of the lights or devices, so if it doesn’t send a state for a light it’ll remain in that state. This can mean commands like “heading to bed” will turn off some devices, not all.

The next iteration of this will probably move the prompt and OpenAI API calls into Node-RED itself, so we can pull the latest state of all devices and include it in the prompt itself. Another option would be make this a Homebridge plugin which would make this a lot easier to pull all known devices and their state automatically.

Update July 2023

One day on my friend couch in LA I decided to evaluate tools like smol-ai/developer and used this article as inspiration — could we replace the no-code Node-RED solution with something easier to configure and maintain? The best outcome came from writing a large explainer prompt into ChatGPT, similar the prompts these tools use (with ReAct + to-do lists) and some handholding during the process.

It worked, almost a little too well, and enabled us to add features such as device auto-discovery, adaptable memory, and much smarter context (now it knows device state before acting, which isn’t possible via plain Shortcuts).

You can check it out at homebridgeai.com

Thanks for reading! If you implement or extend this work I would love to hear about it! Contact/email/twitter details are at tommckenzie.dev