Skip to content

2022

  • 3 min read

More Interactions of Interest

Following up on Interaction of Interest here are a few more things I’ve messed with in the last couple of months that I found to be pretty neat.

Using a Wacom Stylus in the browser — did you know that the HTML 5 pointer event has special properties for pens? I didn’t, but a few months back when I was exploring the benefits of using a wacom tablet with a web app I learned about them. Surprisingly ( to me) a pen just kind of works with many apps for simple events, but allow a lot of custom functionality. I think because of the ubiquity of the mouse and keyboard many apps are built without much thought given to what kind of input device might make the most sense. Of course the individual using the web app needs to possess the device, but I think many interactions can be enhanced through alternative input devices. Pens, knobs, touchpads etc.

Tensorflow.js — I wrote a short article on this one, but in general the short amount of time that I can to spend working with Tensorflow.js left me wanting more. I think that ML has a lot of benefits we are only beginning to discover. The prospect of enabling the use of ML in client web apps and pushing the learning cycle to the user device is really interesting to me.

Forecastle Cashless Wrist Band — Rachael and I recently went to Forecastle. The festival was a lot of fun, but a few weeks before we were set to attend we received wristbands for the event in the mail with registration instructions. Inside the band was an NFC chip that was your ticket into the event. You could also register a card with it to use as a form of payment anywhere in the festival. Having attended quite a few concerts, conventions and festivals this was new to me, but I thought it was awesome. For one it didn’t assume the attendee has a smart device to bring with them, and it also didn’t assume that their smart device would have a good connection to download or open an app, site etc at the event (a common source of issues). What I noticed was that lines to get in were way shorter. No shuffling for tickets, no getting your phone out to open an app, no waiting on face id or entering your passcode. You scan and your in. Payment seemed to have the same benefit of not shuffling for wallets, waiting for you phone/watch wallet to load etc. These bands really streamlined a lot of the wasted time at the end of the line. I hope to see more of these in the future.

Bluetooth Vinyl — This sounds like an oxymoron. I recently picked up a record player that was able to connect via bluetooth to my soundbar and…..it’s awesome. There is something enjoyable about placing the record on the plate, and changing the disk side to hear more as one side comes to a close. That said cable management isn’t fun, and worrying about preamps, speakers just for the player etc wasn’t something I wanted to get into. I’ve been surprisingly happy enjoying the ritual of vinyl with the ease of a bluetooth audio connection.

Pokemon Legends Arceus — another interaction I wrote more about in another article. This game reminded me how much fun exploring a game with somebody else is (I used to play primarily multi-player video games) while also driving home the creativity and care of Nintendo (and for this particular game GameFreak).

  • 2 min read

Interface Discovery

I spend a lot of my days building new tools that individuals use to power their day to day work. Because of this I always keep an eye out for behavior in the apps that I use which inspire me to consider how my apps work. One of the things that has stood out to me over the last year is how some apps introduce let you discover functionality.

Garageband Quick Help — a feature you can enable in Garageband and Logic Pro is quick help. Once enabled it introduces a short tooltip hover over. Considering the amount of options that these applications provide and the depth of the interface being able to simply hover over a button, knob or other part of the interface and immediately find out what it can do is incredibly useful.

Adobe stories — Stories seem like a nice way to share that new functionality is enabled while also creating an index that lets users return to the a story when they are ready to use a feature. Often times when I was using Fresco I would see a new story introducing a new tool, but it wasn’t something I was ready to use yet. I knew it was there though and often times I would return to the story in the next few weeks as I tried out something new. While plenty of apps offer changelogs the presentation and the fact that stories are always right there available in the app is a nice touch.

Emacs/Jetbrains/VS Code and probably your favorite editor — This one has been around for ages, but is still missing in many tools and web apps. Universal search related to functionality. Emacs has had M-x for who knows how long, Jetbrains added Shift + Shift and VS Code has the command palette (not quite the same, or as powerful but good). Being able to toggle into these universal search places and discover options, settings and commands is really valuable and I’ve found plenty of features I never knew existed just by searching a related term.

RTFM — nothing new here, but having picked up some new hobbies and devices over the last couple of years having some docs is super helpful. We often take this for granted, especially if we are building internal tools, but at the end of the day a user manual goes a long way to enabling somebody to read the instructions and be on their way. Bonus points if your docs are built in, indexed and up to date.

  • 3 min read

Pokemon Legends Arceus: Exploring Hisui with my daughter

A couple of months ago I had a pile of games built up that were ready to trade in. When I took them to the shop I saw a new Pokemon game was out, and my daughter is now at the age were some games are becoming fun for her to play. Reading up on the open world nature of Arceus I thought it might be something fun for us to enjoy together. What I didn’t know was just how much fun we would end up having.

From the moment we got to pick our starter (Cyndaquil) we were having fun. My daughter loves all things monsters, loud and colorful so we were off to a good start. Once we powered through the open dialogue as fast as possible (who needs a story when there are monsters?) we were running through the Obsidian Fieldlands discovering all kinds of new Pokemon along the way. My daughter loved it, I was enjoying the nostalgia of seeing familiar faces from my childhood and we were spending time discovering a new world together.

By this point we’ve logged enough hours to explore the far corners of each map, track down alphas and space time distortions, address the rift and we are left tracking down a few lingering Pokedex entries so that we can meet Arceus. It’s been a fun journey around the world, but for me the best part has been seeing the happiness and discovery that my daughter has enjoyed along the way.

Along the way there have been a few things she discovered early on that have endured and become a constant part of our exploration. Anytime we see a shaking rock or tree we have to find out what’s in it. Often times this is met with “just geodude” or “another burmy”, but occasionally it’s an excited shout “WHAT IS THAT!” as Heracross or another unexpected Pokemon pops out.

Bubbles (space-time distortions) have become another ongoing source of entertainment. There is a sense of heightened risk as we are constantly being targeted by the Pokemon spawning in the bubble never knowing what will come next. It’s fun to race around picking up the random items while watching for any bubble denizens we might want to engage.

But most of all I love seeing the stories and interactions my daughter imagines with the non aggressive Pokemon interactions. Each map has a variety of Pokemon or locations were the Pokemon don’t become aggressive as you approach. Probably the most notable for us has been the Alabaster Icelands hot spring where there is often a Machop, Machoke and other Pokemon relaxing and enjoying the water. My daughter loves to run and jump into the water, share snacks (berries) and follow the other Pokemon as they hop and run around. The stories she makes in these moments are amazing and I can only imagine transcend what the designers were imagining as they opted to have less aggressive Pokemon scattered throughout the map. These moments have easily become my favorite part of the game. Maybe in a future Legends the Pokemon in your party will be able to join in these interactions or have non-combative interactions in the wild.

I hadn’t played many games in the last few years, but Legends has been a really fun way to spend time with my daughter. It’s had us sharing ideas, taking turns navigating the world and finding plenty of surprises on the journey to meet Arceus.

P.S. Outside of the fun these fun interactions I have found the game soundtrack to be great. I don’t have any other Pokemon tracks in my playlist, but I’ve added many from this game and I think I may have to go check out the music from previous entries that I haven’t played.

  • 2 min read

Tailored Experiences

A couple of months ago I got to spend a few days hacking on an idea involving client side Tensorflow.js. Having spent much of my career in what I call ML adjacent roles I was really surprised at the opportunities TF.js seem to open up.

One of the things that stood out to me is that the full training and deployment cycle can happen in real time client side. As I was using the tool I was able to load my models starting point and start feeding it data seeing the behavior and predictions from the model change immediately. I could even hop into dev tools to patch and tweak different settings of the model just to see what happened. I didn’t have to wait long cycles, I didn’t have to insert a new notebook cell and disrupt the flow. I was able to launch my web app, load in a checkpoint and go. Based on what I did the model was able to learn in real time (and make some admittedly hilarious mistakes along the way).

This makes me wonder what kind of experiences this is waiting to unlock. I don’t need to send data back to a centralized service for all of my data to be mixed in with the data of other individuals waiting for a new training cycle or model deployment my data was on my device, the model was on my device the update was immediate. Obviously there are risk here (filter bubble, convergence etc) to address, but the opportunity for useful new experiences appears strong.

While I haven’t had much time to explore TF.js since this initial interaction I hope to return to it sometime in the near future. I think that projects like automerge and tensorflow.js represent potential futures where our data is local to us. We can collaborate, share and build all without phoning home or opening the app store if we continue to push what is possible in our browser and devices.

  • 2 min read

Interactions of Interest

Over at Hackaday there is an ongoing series of articles called Inputs of Interest. Last year I wrote my a post Novel Interactions that will end up being the first in an ongoing collection of articles called Interactions of Interest. I personally enjoy building and interacting with new experiences. For those that catch my interest and stay in my memory I’m going to collect and share them.

Sony Linkbuds — while I ultimately couldn’t find a cushion and ear fit that allowed me to listen for more than half an hour I thought the design around these was really creative. Being able to hear your environment while also listening to music, a podcast or book is really nice in a variety of situations. Additionally moving the earbud interactions from being on device to picking up movement from the area near your ear is something I think we might see more of from future devices.

Animoog Z — There are two synth related items on this list. I never used the original Animoog software, but I recently loaded Animoog Z, and it’s been a lot of fun. My daughter and I can sit down tweaking various parameters while launching new sounds into space just to see what we can create next.

Roland 50 — Roland is celebrating 50 years of making awesome devices and tools. In celebration of that they launched a synth web app that is mind-blowing. While this is a lot of fun to create with it is also inspiring to think that this is all running in the browser meaning they could launch it into the world and make it available across device, OS and app store environments with little friction.

Delta — A couple months ago I traveled for work. Along the way I found out I could head home earlier than planned, but I figured making that happen without it becoming a lot of trouble was unlikely. It’s pretty crazy that I was able to take care of bumping my flight up so I could be home with my family all in the Delta app in under 10 minutes without a change fee. Thinking back 10 years ago when I was traveling as a software consultant it’s wild how easy logistics interactions like this have come.

Apple Research App — For the last year I’ve opted in to a few different research studies available in the Apple Research app. Each week or month my phone reminds me I have some questions to answer, and ever so often even has me review the data that I’ve opted into sharing with these studies. What surprises me is how few studies there seem to be. Maybe this is a good thing. Maybe it means there is a lot of effort going into ensuring the quality of these studies, but the way the app works, how transparent it makes things and prompts you to keep in touch makes me wonder if it’s quality, or a lack of (legit) organizations knowing this is out there.

  • 1 min read

I realized something similar a few years back.

I realized something similar a few years back. Discovery is arguably easier than ever, but it's also super easy to fall into just listening to what you know from established playlist.

I have found Bandcamp and some of their newsletters to be a good way to discover new artist and songs.

A few artist from the last 10ish years that you might give a listen to:

  • LEENALCHI
  • Super Future
  • Japanese Breakfast
  • The Weekend

Have fun, there's a lot of good stuff out there!

  • 2 min read

Productive tools can be observed

Last year I started working on a new React app. I was late to the React game, having only started using it with React 16 I had experience with some existing React projects, but this was an opportunity to start something new. Additionally, I was starting out small with one focused use case, but if that was successful [it was!] the app would grow to encompass quite a few screens and tools.

In the process of determining how state would be managed I wanted to test out using useContext and useReducer to see how far I could get before reaching for a state store. For the initial use case this worked great, and to this day I still make use of context for a few key parts of the app. Eventually though as the app grew, and as I wrote more custom reducers I decided it was time to explore my state store options. Along the way I looked at the latest versions of mobx-state-tree , redux (specifically RTK) , relay and others.

Ultimately I ended up choosing Redux Toolkit (for a variety of reasons), because it’s opinionated and easy to observe. The languages we use to solve problems today are not precise and encompassing enough to build systems that cover the universe of execution paths. Because of this it’s important that the tools you use help you understand the system when it behaves in an unexpected manner.

This is pretty trivial in Redux when using Redux Dev Tools. Add in using RTK Query and suddenly you might feel like you have superpowers for understanding the state patterns in your app. With the time travel and state sharing capabilities built right into Redux Dev Tools you can snapshot a session, share it and walk back through each dispatched action in a way that almost feels like magic compared to other debugging experiences I’ve had over the years [writing to the console, re-constructing state manually, browser debugger step by step].

And that’s one of the primary reasons I ended up using Redux in 2022. There are plenty of valid reasons to use other tools, but after all these years I accept that bugs and edge cases are going to happen. Knowing that I want tools that help me explain and understand them, so I can move on to the next step of determining a fix and shipping it.

  • 2 min read

Software Engineering Mythology

Over the last few years I’ve had the opportunity to see how a variety of teams
and companies build software. Along with that (like many others in the field) I’ve kept up with the content coming through various aggregators. Among the many things that every team has to decide is how they are going to actually build and maintain a project(s). We make decisions like:

  • what language to use
  • how to manage dependencies
  • how to generate artifacts
  • how to manage artifacts
  • how to instrument our project
  • how to deliver our project
  • how to document our project

Some of these have grown community standards, some platforms make the decision for us, but often times these decisions are made off a mix of experience and a lot of personal preference. Those preferences I imagine are largely impacted by the industry of personality and marketing that has sprung up around:

  • software education
  • dogma articles
  • self proclaimed experts

Some of the problems with our judgement being so strongly influenced by these voices is that they often lack data for us to place them in our context and judge the merit of what is being offered. Rarely do we know what didn’t work, what came before, the team composition and history, measurements used to determine if something truly was better or more successful. Instead it’s anecdata, stories told through the lens of one reality, comments that evoke strong emotions etc.

This has made me more and more uncomfortable over time. So much time and effort is spent without good data, and we churn and churn and churn. Oddly there are resources published with data from academic sources like ACM, IEEE and industry sources like MS Research and Jetbrains (among many many others). These are not perfect, but they do capture a known context and pair that with data. Maybe the findings have distilled out to our industry over time, but I have a suspicion there is data there that raises questions and counterfactuals to what we extrapolate from ones individual anecdata that turns into the latest hot article/talk.

This isn’t meant to say that the advice we share and the experiences we live are not important. They are incredibly important. Instead I write about this as something I want to go to as a source of education and learning more often. I want to make sure I spend as much time with these sources as I do with articles in my RSS feed. I’m sure I’ll learn something along the way that I haven’t heard before. I hope to find some research spanning groups, companies and project scopes that provides a data analysis of the results on top of the stories of those followed.

ACM
IEEE
Microsoft Research
Jetbrains Research
ISO

  • 1 min read

Without Trust

There is a lof of talk these days about building systems that don’t require trust. On paper this has some appealing characteristics, and throughout history we have seen the issues that arise from unquestionable centralized authority, but right now I think we are also seeing the impacts of missing trust in the analog (physical) realm.

As our physical and digital existence converge and the lines become blurrier I think it’s pertinent to keep in mind the kind of systems and philosophies that we construct in each realm and how they can go on to influence the other.

I have a lot of thoughts and feelings on these subjects, and maybe I’ll go on to explore and write them down in the future, but for now I just simply want to say that I don’t want to live in a world without trust.

  • 4 min read

Novel Interactions

I started my career as a software engineer working on a lot of data visualization projects. After a couple years I switched to working primarily on backend services, and then after doing that for almost 8 years I had the opportunity to start working on frontend projects again. Since switching back I pay more attention to my digital experiences and notices the ones that surprise me or stand out. I wanted to share some of those experiences from 2021.

Apple Pencil + Adobe Fresco + Watercolors

Toward the beginning of last year I started drawing again and came across Fresco on my iPad. While the whole app is impressive the water colors really surprised me. There was something about the first time I chose a water color brush and could see the colors diffusing on the canvas while I was moving my pencil that was really intriguing. Add to that the pressure sensitivity and I continue to find this really interesting.

PokPok

This is a game that my daughter likes to play on iPad from time to time and it really feels like a digital toy box. One of the play areas is a town map where there are tons of interesting interactions. You can move cars, people, animals and have little interactions with the mail, car wash etc. It feels really well made, and like it was made to encourage kids to use their imagination. There are no objectives or dark patterns, just play and it’s great.

GitHub CoPilot

I’ve spent a lot of my career building tools and services for ML projects which has lead me to be critical of many “AI” projects. This year when GitHub CoPilot launched I was able to get into the beta and found out it’s actually a pretty powerful tool. Yes you can find examples of how it can be wrong, and you can find plenty of tales of woe about the bugs it will introduce, but I’ve actually found it to be a really useful coding assistant to the point that I miss it (but can still do my work) when I don’t have it. While it does make some useful code recommendations it also does a great job of helping me write the docs which is worth a lot to me and future team members.

GarageBand on iPhone

I remember getting my first MacBook (white clam shell) and spending a lot of time playing with GarageBand. I was blown away that this was built in for free and could do so much coming from Windows and headless Linux. I had a lot of fun with it, but then slowly I was doing less with music and hadn’t touched GarageBand in years. This year I took a musicianship course with UC Berkeley (through Coursera) which required ear training throughout the week. I used a lot of tools for this, but one that I found to be really fun and interesting was GarageBand on iPhone. After installing it from the AppStore I was able to load up a keyboard that let me practice intervals with some nice sounding samples. From there I went on to explore the various digital instruments and track options. It’s pretty wild just how much is available for creating and interacting with tracks on my phone.

XCloud

Does anybody else remember MSN Zone? I do, and I also remember getting kicked off the dial up connection halfway through a match of Star Wars Battlegrounds because somebody would call the house. Fast forward 20 years and I was able to stream games on my iPad while Rachael and I were in Chicago. Game streaming has been a thing for a while but I really was amazed with the overall experience of the XCloud app. Logging in on the go and loading up Halo Wars 2 in HD with no issues for a couple matches was awesome. Add on being able to see a bit of what was going on with Redux when I did the same thing on my desktop browser and I continue to find XCloud to be really neat.

Spatial Audio

We switched to Apple Music from Spotify last year when they announced the addition of Dolby Atmos/Spatial audio. I remember listening to Hybrid Theory (Linkin Park) first and was blown away. This was an album I had listened to on repeat for years, first with my Sony Walkman disc player and then my RCA MP3 player after that. I felt like I was hearing sounds on tracks for the first time and I continue to go back and check it out week after week. Now I search around for albums I had heard in the past to see what new sounds I might hear while also enjoying what people are doing with spatial audio in games and other mediums.

Listen to Jonathan x Friends - Best of Dolby Atmos + Spatial Audio by Jonathan & Friends on Apple Music. So last year had quite a few new and interesting digital experiences. Hopefully this year has just as many for me to share.