Presently sponsored by: 1Password is a secure password manager and digital wallet that keeps you safe online

I've spent more time IoT'ing my house over the last year than any sane person ever should. But hey, it's been strange times for all of us and it's kept me entertained whilst no longer travelling. Plus, it's definitely added to our lives in terms of the things it enables us to do; see them in part 5 of my IoT unravelled blog series. But it also remains a complete mess and I want to demonstrate why based on some recent experiences:

Let's say you want an IoT light. You want it connected because you want it to come on at a certain time of the day therefore it requires scheduling. You also want to be able to change the colour because hey, that's kinda cool. The light sits on your desk and plugs into mains power and connects wirelessly to your network. An app provided by the device manufacturer controls the schedule, the colour and other features such as the brightness. Now you also want one in your ceiling, but it's different because it's slotting into a standard 92mm cut out. It's still connecting to a mains socket in the roof and it's still connecting to your wifi, but it's a different brand. With a different app. And you have to schedule it differently.

But we're only just getting started here because the next problem is the dependencies you're creating on "the cloud". The manufacturer ships these lights with tiny microprocessors and wifi chips in them at bargain basement prices as a "turnkey" solution; your average consumer picks one up off the shelf, connects it to their wifi, sets up an account and its job done. That account is on "the cloud" and when they're using the custom app, they're communicating with said cloud which is then communicating with the devices. Not all IoT works this way, but a lot does, for example the massively popular Tuya lights I've been putting all over my house.

Pre-pairing everything before they go into the roof: pic.twitter.com/wfHeXrZdqd

— Troy Hunt (@troyhunt) May 24, 2021

And this is where the problems begin.

Ideally, you want local control and local orchestration of all your IoT things and there are several very good reasons for this:

1. Privacy: You may not want to share the state of your connected things with a third party
2. Autonomy: You may not want to be dependent on either the cloud service being up or your internet connection being available
3. Latency: The round trip to the cloud can produce a noticeable delay, especially evident when performing multiple tasks in sequence

The vast majority of consumers will be fine with this because they never expected anything more and frankly, they're still just marvelling at being able to turn on their lights from their phone. But they're experiencing a very watered-down version of IoT that is merely one little self-contained unit. Start pushing the boundaries a bit and things become... a complete and utter mess. Let me explain:

I have 86 Tuya lights in my house (they're not branded as Tuya, but they're all compatible and are controlled by the Tuya app) and they're all surfaced into Home Assistant (HA) via the Tuya integration:

With a bit of mucking around (manually setting the supported features), I can control on / off, colour, warmth and brightness all from within the HA user interface:

I can also control it via YAML which I can use to define scenes and automations:

And finally, I can also control the digital light via a physical switch on the wall as it has a Shelly behind it which sends a message via MQTT to toggle the light.

In short, it's the best of all worlds. Almost - there's still that cloud dependency and there's really only 2 ways around that:

1. Control the existing device locally with the original firmware
2. Flash the device with 3rd party firmware that supports local control

Let's explore these more starting with the first option because it feels like the most low-friction path. There's a dedicated integration for local control of Tuya devices and it's literally just called Local Tuya. In order for it to work, you need to extract both the internal ID of the device and a local key, which is where all the problems begin. You used to be able to pull this out of Tuya's IoT portal but recent changes they've made have completely killed that option (see "!!! UPDATE   !!!" in the description of that video). So that's out, let's instead fire up BlueStacks, install the Smart Life APK, fire up BSTweaker and then pull the keys out of there. I had a false start here because I pulled current versions of everything when what I really needed was an old version of BlueStacks and an old version of the Smart Life APK. Which presented another problem:

Because I don’t know when to give up, I went back to BlueStacks to work out what went wrong. There are **2** Tuya apps, one is TuyaSmart which I’ve paired dozens of lights through. The other is Smart Life which is the one used in the demos where the keys are pulled from it. pic.twitter.com/uAHsmGLq2p

— Troy Hunt (@troyhunt) June 7, 2021

I don't know why there are 2 Tuya apps nor why one is called "Smart Life" and one "TuyaSmart". But fuck it (yes, I'm swearing a lot by now), let's just create a brand new account on Smart Life, pair a new bulb to it then pull out the keys. It'd be hard work to do that for every single bulb, so I'll just do a test one first:

Totally different accounts used on both, so do I unpair every single light from TuyaSmart and manually repair them all to Smart Life? Let's check this all works first so I pair a test light to Smart Life and successfully extract the keys per this vid https://t.co/dWH43btFlq pic.twitter.com/Z9l5gouAvr

— Troy Hunt (@troyhunt) June 7, 2021

Hey, I've got keys! Which don't work in Local Tuya:

The Local Tuya integration finds it on the network by device ID, I fill in the local key, submit and... pic.twitter.com/lyq1wbTICX

— Troy Hunt (@troyhunt) June 7, 2021

"See log for details." Ok then:

FFS. Ok, so let's check the log for some meaningful messaging about what went wrong... pic.twitter.com/s3OJ5EoMke

— Troy Hunt (@troyhunt) June 7, 2021

And even if it did work, I really don't like the idea of unpairing then re-pairing 86 lights one by one.

Let me pause here and reflect on the problem I'll come back to later on: All of us following one of these paths whether it be local control or firmware flashing are attempting to make devices behave in a fundamentally different fashion to how the manufacturer intended them to be used. We're tinkerers trying to solve problems, but we're working against the basic design intent of the devices. It's the same as the problems late last year when TP-Link killed local control of smart plugs:

There were security vulnerabilities on the plug for the local management, the latest firmware version fixed these security issues. It is suggested to use the TP-Link official App KASA to manage the plug. If you have issues, pls feel free to let us know.

— TP-LINK UK (@TPLINKUK) November 17, 2020

The manufacturer is under no obligation to support us tinkerers. I will come back to this because I don't believe it should be that way, but I want to emphasise this point because we should know what we're getting ourselves into here.

So, local control of the out of the box device is out, let's move onto updating the firmware and one approach is to literally solder wires to smart lights as part of the firmware flashing process. That's... messy. And time consuming. And if you screw it up then your smart light isn't just dumb, it's dead. And whilst you might be able to do that with a downlight which has an exposed circuit board, how do you do that with a light globe? So. let's explore the non-soldering route which brings us to flashing ESPHome with tuya-convert, a process which requires the following:

In order to run tuya-convert, you'll need a Linux computer with wifi, plus a second wifi device (such as a smartphone)

Let's pause here a second: If we think about IoT as being something that should be consumable by the masses, just how far down the rabbit hole have we gone once you need a Linux machine and a secondary device just to make your lights behave. This is nuts. But let's do it anyway 🙂

I go buy another Raspberry Pi, set it all up, run tuya-convert and then...

All of this is just different levels of pain. BlueStacks and the Smart Life APK? My Tuya creds don't work. So screw it, just setup a dedicate Pi and use Tuya Convert to flash firmware. Nope, that won't work either: pic.twitter.com/mPbBr0uFZJ

— Troy Hunt (@troyhunt) June 7, 2021

By this stage, my fiancé has rushed upstairs to ask what all the swearing is about. I've also well and truly passed the ROI of trying to gain local control of my lights, so I put that effort to bed and wrote this blog post instead.

Let's move on and instead talk about fixing it:

Devices should enable local control via a baseline set of APIs that remain stable. Lights, for example, should provide dimming, colour and warmth controls in a fashion that is not broken by later updates. Some lights will need to extend on this; the Nanonleaf is able to control light shows across panels so requires functionality beyond what's needed in a single bulb. All of this requires manufacturer buy-in. It's not necessarily hard (have a look at Shelly's fantastic API documentation), but it requires commitment and for that, there needs to be incentive. I'd like to think that the incentive is both to play an active part in contributing to a more interoperable IoT world and financial. The former drives adoption and the latter is realised when the barriers to that adoption are lowered. Which is why I was especially happy to see this recently:

We had a call with @tuyasmart this morning. They are working on an official Home Assistant integration maintained by them.

The beta is already available as a custom component at https://t.co/71bjKySm4X

It connects to their official cloud API. But local access will follow. pic.twitter.com/LmbkMjRo7q

— Home Assistant (@home_assistant) June 29, 2021

This is precisely what we want to see: the vendor themselves playing nice with other platforms and working towards local control. At least that's their indication, but then TP-Link also made indications last year and is yet to deliver so I'll hold off giving Tuya too many kudos for right now. What we need to actively move away from is polyfill solutions like Home Assistant's existing Tuya integration. Don't get me wrong - this is an admiral effort in lieu of Tuya themselves providing APIs - but having Tuya themselves provide APIs (and ideally the Home Assistant integration as well), is precisely where the industry needs to move to.

Lastly, this might seem painful right now (and frankly, it is), but it's also a very exciting time in IoT. It feels like the very early days of the web where everything was a bit of a kludge we hacked together but we made things work and it turned into something amazing. That's where I think we are now with IoT and as infuriating as it often is, it's an exciting time to be a part of it and well and truly worth a few lighting problems here and there 😊