My first commit to on-air shows March 3, 2020. I know that the weeks leading up to that commit I spent some time reading through the TF Lite documentation, playing with Cloudflare Workers K/V and getting my first setup of esp-idf squared away. After that it was off to the races. I outlined my original goal in the planning post. I didn’t quite get to that goal. The project currently doesn’t have a VAD to handle the scenario where I forget to activate the display before starting a call or hangout. Additionally I wasn’t able to train a custom keyword as highlighted in the custom model post. I was however able to get a functional implementation of the concept. I am able to hang the display up, and then in my lab with the ESP-EYEplugged in I can use the wake word visual followed by on/off to toggle the display status.

Recently I spent some time learning how to generate synthetic voices using espeak. After working with the tools to aligning with the Tensorflow keyword models expectations I was ready for training, and to see how well the synthetic data performed. TLDR: not well :)

After getting the display and worker up and running I started down the path of training my model for keyword recognition. Right now I’ve settled on the wake words Hi Smalltalk. After the wake word is detected the model will then detect silence, on, off, or unknown.

Continuing my project with things I know the PyPortal display was up next. Last year I spent a few weeks playing with the portal to make a badge at Gen Con and had a lot of fun with it. Since that time CircuitPython 5 has been released and the portal now expects a few new modules which were easy enough to download and send to the board. The PyPortal makes it incredibly easy to point at an endpoint to fetch data:

After figuring out what I was going to use for my project I started work with things I know. I already had some experience with Cloudflare workers building a home system status page, and Workers K/V makes storing and fetching data quick and easy. I ended up with a simple endpoint that I POST to set a bit after keyword detection, and the PyPortal retrieves that status to determine what to display: