Skip to content

Index

  • 2 min read

An Interlude

I started collecting notes here almost 3 years ago at my first PyCon. I had been working in Python for 3 years at that point and wanted to share some notes with anybody that might find them useful.

Since then I’ve veered into building a homelab, C#, rust, infrastructure, gardening and all kinds of other fun experiences.

More importantly my daughter was born, and is turning 3 this year. Over 2020 I became more and more aware of what I spent time on, and started to evaluate if I wanted to keep spending time on these activities. Many things have fallen to the cutting room floor, and writing up a post each month is one of those. I still do a lot of writing for work, and maybe if something from there manifest itself as a good topic I’ll share it, but I am not going to make myself sit down each month as I have in the past.

I do plan to have short journal snippets I capture each day that I will link to here eventually. Many of them will be software related as that’s what I spend so much time on each day, but all of them won’t be.

So yeah, hopefully the notes here have helped somebody. They will stay up, but I need to give myself permission to let this sit while I spend time elsewhere, maybe to return one day :)

As always if you read something interesting, or find a code snippet that could use expounding send me a message.

Till the next post surfaces 👋

  • 2 min read

Reading Highlights from Q1 2021

I read a lot of books throughout the year, but rarely write anything up after finishing them. While I don’t think I have enough to say about any one book after my first read, I want to capture a sentence or two about them to look back on, and to share with others. If I mention a book here that you want to talk about send me a message.

Q1 2021 List

  • A Philosophy of Software Design — a good read that will impact how you think about writing your code. Many of these ideas have permeated out into general software culture (I think, maybe I’m in a bubble), but I would recommend this.
  • Why we make things and why it matters — an interesting read on crafts and why we should embrace them.
  • Humankind — good, but long at many points
  • Kubernetes Getting Started — as advertised
  • The Unix Administrators Handbook — I had a former manager recommend this book to me. I’ve read it through twice now and I always learn something new.
  • Fluent Python 2nd Edition — A really good book on Python exploring areas that are not always tread, and Luciano does a great job of answering “why” as he goes.
  • Django Crash Course — as advertised, a good alternative or secondary first Django tutorial.
  • A Scoop of Django — A good read. I don’t feel like the answer of why is always explained to the level I would like. It’s hit and miss, sometimes it is, sometimes there is a link to a blog that I may get around to reading one day. A good book none the less.
  • How to Invent Everything — probably my favorite book so far this year. This is a lot of fun and I look forward to reading it with my daughter one day.
  • Think Julia — I’ve heard a bit about Julia and decided to check this out. A pretty good book revisiting a lot of programming first principles in the context of learning them with Julia.
  • Math for Programmers — I really enjoyed this. I try to keep math as a regular reading topic, and I like it when math is demonstrated via programming.
  • 1 min read

Entropy

noun: entropy; plural noun: entropies; symbol: S
  1. a thermodynamic quantity representing the unavailability of a system’s thermal energy for conversion into mechanical work, often interpreted as the degree of disorder or randomness in the system. “the second law of thermodynamics says that entropy always increases with time”
  2. lack of order or predictability; gradual decline into disorder. “a marketplace where entropy reigns supreme”
  3. (in information theory) a logarithmic measure of the rate of transfer of information in a particular message or language.

Everything we build and learn requires constant maintenance and labor.

  • 2 min read

Finding good defaults

At heart many of us are tinkerers. We like to take things apart and see how they work. We enjoy spending hours customizing our tools, scripts and applications. But all of this adds up. It means that our tool works different from all the others. For somebody else coming behind us it may mean piecing together all the flags and tweaks no matter how well we documented or versioned tweaks in our repo, time will bring a divergence between the system and the repo.

A lot of this has set in for me over the years teaching programming, speaking with family about their devices, and onboarding to new code bases as I change teams.

I want to try out something new over the next few years, and that is finding good defaults. If I feel the need to tweak/change a tool out of the box, I’ll try other options. I have some theories around this.

  • Good defaults save us time
  • Don’t require extra conversations for team consensus
  • Help manage complexity
  • Build on expertise
  • Teach us to expect good defaults, and to build our software with good defaults
  • Good defaults help others join in

Against this list I also believe there is a counter culture around defaults wrapped up in programmer/hacker/software engineer reputation that drives us to distrust or question anything we haven’t pulled apart and customized. This creates an extra challenge.

I know that each domains “good defaults” will be different, but that doesn’t mean they don’t exist. My hypothesis is that they can save us time, help us build what matters, and help those who are join in, or come after. Lets find out.

  • 2 min read

Setting up a CI pipeline for Rust in Teamcity

Towards the end of last year I started working on a project in rust that would listen to a message queue and send an email. Additionally it used rocket to expose some diagnostic endpoints to check on the health of the service, change log levels, etc. When starting new projects I default to setting up a build pipeline for them to. For this project I setup pipelines in teamcity which was overall pretty easy, but sharing here for anybody else that may go down this path.

cargo-make

For new projects I like to capture the build, admin and CI steps in a way that makes it convenient for others to run on their local machine. Make and it’s derivatives (cmake, cake, etc) provide a useful task abstraction and Rust has the powerful cargo-make project that lets us capture task and mix together inline simple commands with scripts, dependencies etc.

For this project you can find my cargo make file here. I also experimented with using Powershell for my scripts/wrappers. I’ve been using this in my day job where our projects run on Win, macOS and Linux. Overall I’m pretty happy with the experience, but it is another tool to install and maintain along with various platforms missing support.

cargo test

Rust comes with a build tool and test runner built in via cargo. Running test is easy out of the box, but I needed to make use of a couple tools to get the cargo test output into a format that a CI tool parses. I ended getting test and coverage data in the junit and lcov formats that way various tools and platforms can be used across time and projects.

Teamcity

With those tools orchestrated via cargo make it's time to setup the build and test steps in Teamcity. Overall the process was pretty easy, but I ran into a couple bumps I'll highlight.

  • The cargo step doesn't support custom commands, so I don't use that by default
  • I wrote CI.ps1 as a wrapper to use in each step.
  • Enable the xml-report-plugin And with those two things the pipeline is ready to go. From there you may want to add your own environment variables, plugin, agent deps etc.

Next steps

With this pipeline up an running the next steps are:

  • Setup build caching with something like sccache
  • Work on local and CI build times
  • This has been written about a number of times I would need to make both of these better before taking the project further. As the project grows these would only get worse, and make the project unpleasant for others to work on.

Done

That’s it for now. I learned a lot along the way about Rust, cargo, and hooking it up with Teamcity. I’m not sure I’ll have a write up on artemis anytime soon. It was a good project, but I ultimately took another path. Hopefully this helps somebody, and as always feel free to reach out.

  • 2 min read

The joy of repair

A few weeks ago the Z Upper Right Assembly broke on my Mini 2. At first, I wasn’t quite sure what was wrong I only knew that the tool head couldn’t raise on the right-hand side. In an email to Adrian (at Lulzbot) we figured out the part had a hairline crack. In a year without covid I could have gone down to LVL1 to make a replacement part, but not this year. Luckily Lulzbot was easy to get a print from, and really fast too (kudos to them, and that’s nice to know for the future).

With this being my first repair I was a bit nervous. I had never taken my mini apart or tinkered too much instead opting for that with my prints. The good news is Lulzbot has amazing documentation for each printer that make repairs relatively straight forward. Beginning to end taking things apart and getting them back together per the doc took me an hour, and then it was go time. My first print was rocktopus which came out great. Seeing the right-hand axis work as expected was a huge relief, and gives me some confidence that I could do more for in the realm of mods and repairs in the future.

Why does this matter? Because if you can’t fix it, you don’t own it, and I was blown away how easy it was to fix my Lulzbot. On top of that it was fun. Outside of gardening I don’t have the opportunity to just fix stuff with my hands that much, and I miss that. Something to look into this year maybe?

  • 2 min read

Reading highlights from Q4 2020

I read a lot of books this year, but rarely write anything up after finishing them. While I don’t think I have enough to say about any one book after my first read, I want to capture a sentence or two about them to look back on, and to share with others. If I mention a book here that you want to talk about send me a message.

Q4 2020 List

Zen and the Art of Motorcycle Maintenance

  • I’ve been really interested in the idea of quality and how we achieve it in what we build. Everybody uses a different definition of “good”, and I enjoyed the exploration of the topic in this book.

Blockchain Chicken Farm

  • Having grown up in rural kentucky and working in industrial chicken farming, then pivoting to writing software after high school this was a fantastic read. A lot to reflect on, the shared humanity, and the audacity of the software industry shined beginning to end.

Subprime Attention Crisis

  • A good exploration of the dangers in the ad based internet economy. While the ad industry has made a lot of tools available to those who couldn’t afford them outright, I think we can do better than the current ad economy. While shrinking sectors can hurt, I think we need to be careful not to accidentally lead individuals to believe we need to maintain the ad economy at the invasive scale we are at today.

Understanding Computation

  • As advertised. Computational theory using Ruby. I enjoyed the read and will probably revisit Part II sometime. It is a big book covering a wide range of topics and I don’t think I internalized Part II enough.

The Hardware Hacker

  • A collection of articles from “bunnie” relaying his experience and some philosophies while manufacturing in the open. The term shanzhai has popped up here and in other reading over the last few years.

Angular Development with Typescript, Second Edition

  • What it sounds like. Reading this for work, and it was helpful as I got going with Angular.

Programming Typescript

  • What it sounds like. A good read after going through the docs at typescriptlang.org.

Vader Down

  • Vader being Vader. Fun short read.

Shadowfall

  • This popped up as new at the library, and I was a few chapters in when I realized I had missed the first book in a series. So far I like it. It’s in the same vein as rogue squadron, and the author is working to write characters in a complicated environment. Curious to see how the series wraps up and if the character arcs land.
  • 1 min read

Org templates and checklist

Last year I read the “The Checklist Manifesto” about the outsized impact a checklist can have on an individual and teams.

I also started using org-mode to keep a daily journal of what I'm working on, design notes, todos etc. Having spent a few weeks with org-journal I decided it was time to create my own templates. For my first template a checklist seemed like an easy target. I made a few like PR Review, and New Service which can be found in my dotfiles along with their bindings.

True to the goal, I’ve got a lot more consistent in my reviews and with capturing review artifacts. The service checklist has been turned into a dotnet template that saves even more time.

Yay checklist!

  • 2 min read

Use data structures for your business logic

A few months ago I was reviewing a PR that handled relationships between entities. As I was working through the code I started to notice a pattern that made me go back to the original feature ticket for a quick review of the acceptance criteria. As I suspected there was a list of around 10 “if this then that” scenarios detailed, all of which manifested as conditions in the code. Grabbing a pen and paper I started to draw out the criteria and as I suspected all the scenarios were captured by relationships and operations for a Tree.

Going back with this information I paired with the team on an update to the PR where we reduced the amount of conditions tied directly to the business domain, and refactored names so that future maintainers could interact with the code understanding a tree, but maybe not understanding all the business logic around the entities.

in case it’s helpful the C5 project has some collections not found in the .NET Standard library for interacting with Trees. In general an interesting project I’m glad I learned about.A similar opportunity emerged on the same project when we needed to make sure a value was unique over a series of operations. In this scenario while working on a collection of objects we were able to use a HashSetto exit if Add returned false instead of setting up a LINQ query. This resulted in less nesting, less code, and a simplified condition.

The Point

The reason I am writing this is that we should be using data structures to represent the business logic of our applications. This seems obvious, but too often I have seen implementations brute force conditions leaving data structures as an optimization, or a concern for “technical” projects. While we can use a series of conditions and predicates to meet requirements in a crude way, using data structures provides an abstraction that can elevate terse business logic to a construct future maintainers can derive extra meaning from.

  • 2 min read

Self Hosting

Over the last few years I built up a sprawling list of dependencies for my home project and blog workflow. Earlier this year I decided it was time to cut down on that list and host my service dependencies locally where I could. While it took me a while I reached a point where I no longer tweak the setup week to week and decided it was time to write up the process.

A quick list of the tools I used for orchestration:

  • shell
  • DNS
  • Traefik2
  • docker/docker-compose
  • alpine linux

(D)DNS

The first thing I needed to do was make my services easy to reach local and remote. Since this is all running behind my home router that also means that my IP can change from time to time. To handle this I made use of Gandi’s DNS API, and setup a shell script to run with cron on my router to keep my DNS records up to date. With DNS ready I moved on to Traefik.

Traefik

Traefik is a really nice routing/proxy service that can inspect container labels and setup route forwarding while handling certificate management, traffic metrics and more. The main callout (other than what you will find in the docs ) is to keep an eye on what version you are using versus what others used in examples, and that non http based traffic (for instance ssh) requires a little more setup. Beyond that Traefik has been really nice to use and made adding/removing various services easy when coupled with docker.

docker-compose

While k8s is the current hot orchestration tool I wanted to keep things simple. I don’t have a need to cluster any of my home tools, and while distributed systems are interesting they also require a lot of work. I left those at my day job and use compose + duplicity for my home setup. This makes service management easy, the labels allow traefik to detect and handle traffic management while my duplicty ensures I won’t lose much work and can quickly restore my data and restart any services in a few minutes on any box with docker.

Services

A quick list of the services I’m hosting:

  • git
  • cgit
  • minio
  • teamcity
  • youtrack
  • rust home services API

The service management can be found here.

Wrap Up

I’ve started to self host a few times in the past and backed away. This time I think it’s here to stay. With my current setup I’m not worried about what happens when something crashes, certificate management is automated away and everything just works. I’ve linked to my orchestration code above, but if you have any questions, or suggestions send them my way. If you are starting out on your own self hosted setup, good luck, have fun it’s easier now than ever and I imagine it will continue to get better.