Skip to content

Index

Notes from a year of working with Jack Audio Connection Kit

Last year I became the lead engineer on a new project. After joining the project I spent a lot of time helping users and engineers debug JACK audio behavior (although JACK typically wasn't the cause of the observed issue). While I had used Core Audio in the past I wasn't as familiar with JACK at the time, so I thought I would write down a few of the things I have learned to check first when somebody tells me they are having an issue with JACK. These are specific to running JACK on Linux, in my case Ubuntu.

flowchart LR
    A[Does JACK server start?] --> B{No}
    B --> C[What is your clock source, and is it valid?]
    C --> D{Does your device provide JACK a clock?}
    D --> E{"Does your device have an internal clock,\n or does it rely on an external clock\n (for instance a PCI MADI card setup)?"}
    C --> F{Are all of your cables connected?}
    F --> G{Are they in the right direction?}
    F --> H{Are they all showing healthy conditions on\n their LED or other connection health indicator?}
    B --> I[Does ALSA recognize your sound card/interface\n as a capture and/or playback device?]
    I --> J{No}
    J --> K{"How is the device connected (USB, PCI, etc),\n and do you need any additional drivers?"}
    J --> L{"Using the correct tools (lsusb, etc)\n does the kernel recognize the device is present?\n If not you're debugging hardware."}
    I --> M{Yes}
    M --> N{Does ALSA recognize that the device supports\n the sample rate you want to run the JACK server at?}
    M --> O{Do you need to customize any channel or other\n settings with alsa mixer to have the correct IO setup?}
    M --> P{Can you use alsa utilities to send or\n receive signal to the device?}
    P --> Q["If your clock source is valid, and ALSA\n recognizes the devices and can route signal in/out of the device\n (depending on playback and capture capabilities and requirements)\n then we probably have a JACK settings issue.\n If you don't verify the clock and ALSA\n settings first though you may be looking in the wrong spot for the issue."]
    C --> R{Clock is valid and ALSA can use the device}
    R --> S{Are you starting JACK in daemon mode or dbus?}
    S --> T{If daemon mode:}
    T --> U{Is there anything blocking the daemon from starting?\n For instance another jack server that is already\n running via pulseaudio or another process?}
    R --> V{Are you passing the right hardware identifier\n to JACK to select the correct playback/capture device\n that the process will use with the server?}
    R --> W{Are you starting the server with a sample rate,\n frame size and period supported by the device?}
    W --> X{Use ALSA tools or qjackctl to review device settings\n and configuration options.}
    R --> Y{What does JACK tell you if you start the process\n with the `--verbose` flag?}

That's a lot to check, and I employ various ALSA cli utilities along with JACK tools like qjackctl to help collect information as I iterate through the items above until I get to a state where the JACK server will start. Once I get the JACK server started if errors are reported it tends to fall into a couple categories (I'm sure there are more, this is just based on my experience so far):

  1. JACK client issues
    1. This is a pretty broad space since really a jack client can represent just about any application/algorithm that you want to wrap as a jack client and have audio or midi in/out port support. A few things I've seen:
      1. The application runs at a fixed rate that isn't compatible with the rate the server is running at.
        1. You will either need to update the client, run the server at a different rate, or do sample rate conversion.
      2. The application parses JACK server output information and doesn't behave correctly when the server runs in verbose mode.
        1. There is probably a better way to get the information the client needs without parsing jack server output.
      3. The MIDI messages that the application receives are not parsed correctly.
        1. Use MIDI utilities to check the message format/structure and either fix the sending or receiving application code.
      4. Resource errors - these tend to be addressable with standard tools (writing tests, profiling the application, static analysis, etc).
        1. Memory errors.
        2. Callback code doesn't execute within the allotted time.
        3. Improper port management.
        4. Logic bugs.
  2. Unstable clock
    1. This one is a little tricky. If the jack server starts, but loses the clock source you can check a few things.
      1. Is the device still connected correctly, and if it's external did it lose power?
      2. Check syslogs and dbus. Did the hardware disconnect and reconnect?
      3. If the device clock is external does the device have any logs that you should review?
      4. If none of the points above expose any useful information then you will probably need to do deeper monitoring, and possibly troubleshooting at the driver/device level.
        • Does the error happens on a set time interval
        • Can you replicate it on a second machine
        • What debug information is available at the driver level

Working with and building JACK applications has been a really interesting experience. Making sure everything aligns as expected from hardware through software has taught me a lot. Maybe some of this will help somebody else if they are debugging JACK workflows for the first time.

More Interactions of Interest

Following up on Interaction of Interest here are a few more things I’ve messed with in the last couple of months that I found to be pretty neat.

Using a Wacom Stylus in the browser — did you know that the HTML 5 pointer event has special properties for pens? I didn’t, but a few months back when I was exploring the benefits of using a wacom tablet with a web app I learned about them. Surprisingly ( to me) a pen just kind of works with many apps for simple events, but allow a lot of custom functionality. I think because of the ubiquity of the mouse and keyboard many apps are built without much thought given to what kind of input device might make the most sense. Of course the individual using the web app needs to possess the device, but I think many interactions can be enhanced through alternative input devices. Pens, knobs, touchpads etc.

Tensorflow.js — I wrote a short article on this one, but in general the short amount of time that I can to spend working with Tensorflow.js left me wanting more. I think that ML has a lot of benefits we are only beginning to discover. The prospect of enabling the use of ML in client web apps and pushing the learning cycle to the user device is really interesting to me.

Forecastle Cashless Wrist Band — Rachael and I recently went to Forecastle. The festival was a lot of fun, but a few weeks before we were set to attend we received wristbands for the event in the mail with registration instructions. Inside the band was an NFC chip that was your ticket into the event. You could also register a card with it to use as a form of payment anywhere in the festival. Having attended quite a few concerts, conventions and festivals this was new to me, but I thought it was awesome. For one it didn’t assume the attendee has a smart device to bring with them, and it also didn’t assume that their smart device would have a good connection to download or open an app, site etc at the event (a common source of issues). What I noticed was that lines to get in were way shorter. No shuffling for tickets, no getting your phone out to open an app, no waiting on face id or entering your passcode. You scan and your in. Payment seemed to have the same benefit of not shuffling for wallets, waiting for you phone/watch wallet to load etc. These bands really streamlined a lot of the wasted time at the end of the line. I hope to see more of these in the future.

Bluetooth Vinyl — This sounds like an oxymoron. I recently picked up a record player that was able to connect via bluetooth to my soundbar and…..it’s awesome. There is something enjoyable about placing the record on the plate, and changing the disk side to hear more as one side comes to a close. That said cable management isn’t fun, and worrying about preamps, speakers just for the player etc wasn’t something I wanted to get into. I’ve been surprisingly happy enjoying the ritual of vinyl with the ease of a bluetooth audio connection.

Pokemon Legends Arceus — another interaction I wrote more about in another article. This game reminded me how much fun exploring a game with somebody else is (I used to play primarily multi-player video games) while also driving home the creativity and care of Nintendo (and for this particular game GameFreak).

Interface Discovery

I spend a lot of my days building new tools that individuals use to power their day to day work. Because of this I always keep an eye out for behavior in the apps that I use which inspire me to consider how my apps work. One of the things that has stood out to me over the last year is how some apps introduce let you discover functionality.

Garageband Quick Help — a feature you can enable in Garageband and Logic Pro is quick help. Once enabled it introduces a short tooltip hover over. Considering the amount of options that these applications provide and the depth of the interface being able to simply hover over a button, knob or other part of the interface and immediately find out what it can do is incredibly useful.

Adobe stories — Stories seem like a nice way to share that new functionality is enabled while also creating an index that lets users return to the a story when they are ready to use a feature. Often times when I was using Fresco I would see a new story introducing a new tool, but it wasn’t something I was ready to use yet. I knew it was there though and often times I would return to the story in the next few weeks as I tried out something new. While plenty of apps offer changelogs the presentation and the fact that stories are always right there available in the app is a nice touch.

Emacs/Jetbrains/VS Code and probably your favorite editor — This one has been around for ages, but is still missing in many tools and web apps. Universal search related to functionality. Emacs has had M-x for who knows how long, Jetbrains added Shift + Shift and VS Code has the command palette (not quite the same, or as powerful but good). Being able to toggle into these universal search places and discover options, settings and commands is really valuable and I’ve found plenty of features I never knew existed just by searching a related term.

RTFM — nothing new here, but having picked up some new hobbies and devices over the last couple of years having some docs is super helpful. We often take this for granted, especially if we are building internal tools, but at the end of the day a user manual goes a long way to enabling somebody to read the instructions and be on their way. Bonus points if your docs are built in, indexed and up to date.

Pokemon Legends Arceus: Exploring Hisui with my daughter

A couple of months ago I had a pile of games built up that were ready to trade in. When I took them to the shop I saw a new Pokemon game was out, and my daughter is now at the age were some games are becoming fun for her to play. Reading up on the open world nature of Arceus I thought it might be something fun for us to enjoy together. What I didn’t know was just how much fun we would end up having.

From the moment we got to pick our starter (Cyndaquil) we were having fun. My daughter loves all things monsters, loud and colorful so we were off to a good start. Once we powered through the open dialogue as fast as possible (who needs a story when there are monsters?) we were running through the Obsidian Fieldlands discovering all kinds of new Pokemon along the way. My daughter loved it, I was enjoying the nostalgia of seeing familiar faces from my childhood and we were spending time discovering a new world together.

By this point we’ve logged enough hours to explore the far corners of each map, track down alphas and space time distortions, address the rift and we are left tracking down a few lingering Pokedex entries so that we can meet Arceus. It’s been a fun journey around the world, but for me the best part has been seeing the happiness and discovery that my daughter has enjoyed along the way.

Along the way there have been a few things she discovered early on that have endured and become a constant part of our exploration. Anytime we see a shaking rock or tree we have to find out what’s in it. Often times this is met with “just geodude” or “another burmy”, but occasionally it’s an excited shout “WHAT IS THAT!” as Heracross or another unexpected Pokemon pops out.

Bubbles (space-time distortions) have become another ongoing source of entertainment. There is a sense of heightened risk as we are constantly being targeted by the Pokemon spawning in the bubble never knowing what will come next. It’s fun to race around picking up the random items while watching for any bubble denizens we might want to engage.

But most of all I love seeing the stories and interactions my daughter imagines with the non aggressive Pokemon interactions. Each map has a variety of Pokemon or locations were the Pokemon don’t become aggressive as you approach. Probably the most notable for us has been the Alabaster Icelands hot spring where there is often a Machop, Machoke and other Pokemon relaxing and enjoying the water. My daughter loves to run and jump into the water, share snacks (berries) and follow the other Pokemon as they hop and run around. The stories she makes in these moments are amazing and I can only imagine transcend what the designers were imagining as they opted to have less aggressive Pokemon scattered throughout the map. These moments have easily become my favorite part of the game. Maybe in a future Legends the Pokemon in your party will be able to join in these interactions or have non-combative interactions in the wild.

I hadn’t played many games in the last few years, but Legends has been a really fun way to spend time with my daughter. It’s had us sharing ideas, taking turns navigating the world and finding plenty of surprises on the journey to meet Arceus.

P.S. Outside of the fun these fun interactions I have found the game soundtrack to be great. I don’t have any other Pokemon tracks in my playlist, but I’ve added many from this game and I think I may have to go check out the music from previous entries that I haven’t played.

Tailored Experiences

A couple of months ago I got to spend a few days hacking on an idea involving client side Tensorflow.js. Having spent much of my career in what I call ML adjacent roles I was really surprised at the opportunities TF.js seem to open up.

One of the things that stood out to me is that the full training and deployment cycle can happen in real time client side. As I was using the tool I was able to load my models starting point and start feeding it data seeing the behavior and predictions from the model change immediately. I could even hop into dev tools to patch and tweak different settings of the model just to see what happened. I didn’t have to wait long cycles, I didn’t have to insert a new notebook cell and disrupt the flow. I was able to launch my web app, load in a checkpoint and go. Based on what I did the model was able to learn in real time (and make some admittedly hilarious mistakes along the way).

This makes me wonder what kind of experiences this is waiting to unlock. I don’t need to send data back to a centralized service for all of my data to be mixed in with the data of other individuals waiting for a new training cycle or model deployment my data was on my device, the model was on my device the update was immediate. Obviously there are risk here (filter bubble, convergence etc) to address, but the opportunity for useful new experiences appears strong.

While I haven’t had much time to explore TF.js since this initial interaction I hope to return to it sometime in the near future. I think that projects like automerge and tensorflow.js represent potential futures where our data is local to us. We can collaborate, share and build all without phoning home or opening the app store if we continue to push what is possible in our browser and devices.

Interactions of Interest

Over at Hackaday there is an ongoing series of articles called Inputs of Interest. Last year I wrote my a post Novel Interactions that will end up being the first in an ongoing collection of articles called Interactions of Interest. I personally enjoy building and interacting with new experiences. For those that catch my interest and stay in my memory I’m going to collect and share them.

Sony Linkbuds — while I ultimately couldn’t find a cushion and ear fit that allowed me to listen for more than half an hour I thought the design around these was really creative. Being able to hear your environment while also listening to music, a podcast or book is really nice in a variety of situations. Additionally moving the earbud interactions from being on device to picking up movement from the area near your ear is something I think we might see more of from future devices.

Animoog Z — There are two synth related items on this list. I never used the original Animoog software, but I recently loaded Animoog Z, and it’s been a lot of fun. My daughter and I can sit down tweaking various parameters while launching new sounds into space just to see what we can create next.

Roland 50 — Roland is celebrating 50 years of making awesome devices and tools. In celebration of that they launched a synth web app that is mind-blowing. While this is a lot of fun to create with it is also inspiring to think that this is all running in the browser meaning they could launch it into the world and make it available across device, OS and app store environments with little friction.

Delta — A couple months ago I traveled for work. Along the way I found out I could head home earlier than planned, but I figured making that happen without it becoming a lot of trouble was unlikely. It’s pretty crazy that I was able to take care of bumping my flight up so I could be home with my family all in the Delta app in under 10 minutes without a change fee. Thinking back 10 years ago when I was traveling as a software consultant it’s wild how easy logistics interactions like this have come.

Apple Research App — For the last year I’ve opted in to a few different research studies available in the Apple Research app. Each week or month my phone reminds me I have some questions to answer, and ever so often even has me review the data that I’ve opted into sharing with these studies. What surprises me is how few studies there seem to be. Maybe this is a good thing. Maybe it means there is a lot of effort going into ensuring the quality of these studies, but the way the app works, how transparent it makes things and prompts you to keep in touch makes me wonder if it’s quality, or a lack of (legit) organizations knowing this is out there.

I realized something similar a few years back.

I realized something similar a few years back. Discovery is arguably easier than ever, but it's also super easy to fall into just listening to what you know from established playlist.

I have found Bandcamp and some of their newsletters to be a good way to discover new artist and songs.

A few artist from the last 10ish years that you might give a listen to:

  • LEENALCHI
  • Super Future
  • Japanese Breakfast
  • The Weekend

Have fun, there's a lot of good stuff out there!

Productive tools can be observed

Last year I started working on a new React app. I was late to the React game, having only started using it with React 16 I had experience with some existing React projects, but this was an opportunity to start something new. Additionally, I was starting out small with one focused use case, but if that was successful [it was!] the app would grow to encompass quite a few screens and tools.

In the process of determining how state would be managed I wanted to test out using useContext and useReducer to see how far I could get before reaching for a state store. For the initial use case this worked great, and to this day I still make use of context for a few key parts of the app. Eventually though as the app grew, and as I wrote more custom reducers I decided it was time to explore my state store options. Along the way I looked at the latest versions of mobx-state-tree , redux (specifically RTK) , relay and others.

Ultimately I ended up choosing Redux Toolkit (for a variety of reasons), because it’s opinionated and easy to observe. The languages we use to solve problems today are not precise and encompassing enough to build systems that cover the universe of execution paths. Because of this it’s important that the tools you use help you understand the system when it behaves in an unexpected manner.

This is pretty trivial in Redux when using Redux Dev Tools. Add in using RTK Query and suddenly you might feel like you have superpowers for understanding the state patterns in your app. With the time travel and state sharing capabilities built right into Redux Dev Tools you can snapshot a session, share it and walk back through each dispatched action in a way that almost feels like magic compared to other debugging experiences I’ve had over the years [writing to the console, re-constructing state manually, browser debugger step by step].

And that’s one of the primary reasons I ended up using Redux in 2022. There are plenty of valid reasons to use other tools, but after all these years I accept that bugs and edge cases are going to happen. Knowing that I want tools that help me explain and understand them, so I can move on to the next step of determining a fix and shipping it.

Software Engineering Mythology

Over the last few years I’ve had the opportunity to see how a variety of teams
and companies build software. Along with that (like many others in the field) I’ve kept up with the content coming through various aggregators. Among the many things that every team has to decide is how they are going to actually build and maintain a project(s). We make decisions like:

  • what language to use
  • how to manage dependencies
  • how to generate artifacts
  • how to manage artifacts
  • how to instrument our project
  • how to deliver our project
  • how to document our project

Some of these have grown community standards, some platforms make the decision for us, but often times these decisions are made off a mix of experience and a lot of personal preference. Those preferences I imagine are largely impacted by the industry of personality and marketing that has sprung up around:

  • software education
  • dogma articles
  • self proclaimed experts

Some of the problems with our judgement being so strongly influenced by these voices is that they often lack data for us to place them in our context and judge the merit of what is being offered. Rarely do we know what didn’t work, what came before, the team composition and history, measurements used to determine if something truly was better or more successful. Instead it’s anecdata, stories told through the lens of one reality, comments that evoke strong emotions etc.

This has made me more and more uncomfortable over time. So much time and effort is spent without good data, and we churn and churn and churn. Oddly there are resources published with data from academic sources like ACM, IEEE and industry sources like MS Research and Jetbrains (among many many others). These are not perfect, but they do capture a known context and pair that with data. Maybe the findings have distilled out to our industry over time, but I have a suspicion there is data there that raises questions and counterfactuals to what we extrapolate from ones individual anecdata that turns into the latest hot article/talk.

This isn’t meant to say that the advice we share and the experiences we live are not important. They are incredibly important. Instead I write about this as something I want to go to as a source of education and learning more often. I want to make sure I spend as much time with these sources as I do with articles in my RSS feed. I’m sure I’ll learn something along the way that I haven’t heard before. I hope to find some research spanning groups, companies and project scopes that provides a data analysis of the results on top of the stories of those followed.

ACM
IEEE
Microsoft Research
Jetbrains Research
ISO

Without Trust

There is a lof of talk these days about building systems that don’t require trust. On paper this has some appealing characteristics, and throughout history we have seen the issues that arise from unquestionable centralized authority, but right now I think we are also seeing the impacts of missing trust in the analog (physical) realm.

As our physical and digital existence converge and the lines become blurrier I think it’s pertinent to keep in mind the kind of systems and philosophies that we construct in each realm and how they can go on to influence the other.

I have a lot of thoughts and feelings on these subjects, and maybe I’ll go on to explore and write them down in the future, but for now I just simply want to say that I don’t want to live in a world without trust.