Projects
Personally I find peoples’ “programming origin story” to be valuable in understanding their perspective, so here’s some side projects that I’ve hacked on over the years, in roughly reverse chronological order. This list is definitely incomplete, but you’ll get the idea.
Relatedly, I’ve also given a decent amount of talks and taken a couple of online courses to learn new things.
Factorio-A11y, an accessibility mod (Lua)
Factorio is a very heavily mouse and keyboard oriented game, which precludes people who are unable to effectively use a mouse and keyboard from playing, so I started factorio-a11y, an accessibility mod for Factorio. It provides alternative means to accomplish key actions in the game via a mix of hotkeys and console commands intended to be triggered by voice control, joysticks/controllers, and eye/head-tracking equipment.
Voice control of arbitrary software (Python/Dragonfly)
I developed a set of voice commands which allow me to work without using my hands at all; keyboard interaction and mouse movements are performed by scripts I wrote using Dragonfly, powered by Nuance’s Dragon NaturallySpeaking for the actual voice recognition. The details of how it works are relatively complex, but the core idea is that Dragonfly provides a mechnaism to define a grammar in an EBNF-like domain specific language, then I defined the grammar and arbitrary python code which should be run for each recognised element.
This project was interesting because it gave me a whole new perspective on software accessibility. Missing keyboard shortcuts are annoying for a regular user, but for an impaired user, missing keyboard shortcuts mean bespoke development is required to expose the same functionality - or perhaps the task can’t be done at all.
It was also a great experiment in human computer interaction. As far as I could tell, there had been significant research into voice recognition for arbitrary dictation, and a decent amount of research into voice-based controls for non-technical users (e.g. Alexa), but there was very little serious research or development into voice control for programmers and power users. I realized that the tradeoffs of voice control are quite different from those of a keyboard and mouse: many more possible inputs which are easier to remember than keyboard shortcuts (I have at least 500 commands in my grammar), but at the cost of higher latency and some misrecognitions.
Additionally, there are interesting internal tradeoffs in grammar design: longer commands are more reliable, but slower and less convenient (although usually still faster than typing out a word), short commands also need to balance phonetic dissimilarity with other commands vs their own memorability, the complexity of the grammar itself needs to be managed to keep recognition times under a second, etc.
As of 2019 I still use and work on this project, as the interaction mechanism is well suited to certain tasks and is a nice change from keyboard and mouse, but for my own convenience and privacy I’ve kept my grammars and accompanying code private.
Jess, a JSON pager (Rust)
jess
is less
but for JSON data; it renders JSON data in a tabular format, and allows interacting with it via a text-based user interface. For example, if an object contains a value which is itself an object, you can select that value and “zoom in” on it to see more details. It also supports tailing files for further input (like less +F foo.txt
), and incrementally parsing concatenated JSON files (object by object, pausing at the first incomplete object).
I started it because I was frustrated with how unreadable JSON logs are by default, particularly outside of specialized tools such as Kibana/Splunk. With a bit of effort you can format the JSON to make it more readable, but then log lines often take up 10+ lines for a single conceptual line of log output, making it annoying to scan logs - so you’re back to grepping through them, in which case you might as well be outputting plain logs in the first place.. but then you lose your structured logs, argh! Along the way, the project increased in scope to make it also useful for exploring static large JSON data sets.
This project is still in on-again, off-again (private) development. Eventually, this project should integrate with JQ and/or JMESPath; ideally it should eventually be flexible and clever enough to supplant special-case tools such as Bunyan’s CLI.
Rocksy, a development proxy (Rust)
While playing around with Elm, I realized that Elm didn’t have a nice way to proxy some requests to the Elm development server while other requests were routed to the backend server of choice. In Webpack-land, this problem is solved by the webpack dev server supporting a proxy option, or it could be solved separately by setting appropriate CORS headers, but I didn’t want to introduce Webpack or risk development-only CORS headers accidentally being sent in production. I also didn’t want to run nginx or another such full blown proxy just for my messing around.
So, I did significantly more work to generalize Webpack dev server’s proxy into a solution that’s reusable by any front-end and back-end stack. I wrote Rocksy in Rust using Hyper and Rust’s then-nascent future
s support, and it works well enough now that I haven’t had to touch it since.
Decider, a decision-making web app (Typescript/React)
As an excuse to get more familiar with Typescript, I came up with a novel twist on the standard “weighted pros & cons on a legal pad” decision making technique, and implemented about 80% of it in Typescript and React.
BeNice, a comment sentiment analyser (Python)
I helped my data scientist wife develop a small-scale neural-network-based web API for classifying online comments as friendly or mean, based on sample data scraped from Reddit’s API. Frankly this barely belongs on this list - all I did was provide guidance on design, and help her out when she got stuck using Flask, and all the interesting data stuff (building & validating the classification model) was done by her.
Sherlock-thread, a debugging tool (Rust)
Sherlock-thread is an experimental debug-tooling project for Linux written in Rust, intended to force applications to include thread identifiers whenever they output a line of log information. It works by replacing the symbols for writing strings to file descriptors at application load time (using LD_PRELOAD
), and then intercepting write calls which are writing to stdout/stderr, modifying them to add the thread identifier.
It works fairly reliably thanks to glibc almost always being dynamically linked (partly due to its GPL license), but it does not work at all for programs which statically link a C standard library (e.g. musl often gets statically linked) or programs which avoid calling C standard library functions (e.g. anything written in Go). However, at the time I did this project, Rust did not support calling varargs functions, so it was not possible to complete this project (unless I rewrote it into C or similar), so I didn’t waste time polishing this project.
Logprof, a log-based profiler (Rust)
For an Atlassian hackathon, I wrote logprof, which is a simple profiling tool based on analyzing the speed at which logs are received (or existing timestamps on logs). The gist is that if you have a long running process (e.g. a build) that emits logs, this tool lets you find where your time went by highlighting which bits are slow (optionally via sparkline-graph-like visualizations), which is the first step to speeding up that long running process.
Logprof’s README has several detailed examples showing more usage.
Maven-nailed, a speed boost to Maven (Java)
Frustrated with Maven’s 1-3 second startup delay impacting my development loop, I forked maven-nailgun as maven-nailed, then updated it to add support for the latest version of Maven used by Atlassian at the time, added tests, and made it easier to install for people (important because it required patching Maven!), with the idea of getting other developers at Atlassian to adopt it too.
Board game turn timer (JavaScript)
I wrote & published a super-simple web app for running on a phone or table, intended to be used as a turn timer for boardgames; functionally it’s similar to a chess clock, but for up to 4 players.
Eclipse-multicursor, a plugin for the Eclipse IDE (Java)
Inspired by Sublime Text’s multiple cursors, I took a stab at implementing multicursor editing for Eclipse. My approach was a straightforward but somewhat limited approach using Eclipse’s existing LinkedMode APIs. I also prototyped a more flexible approach, but unfortunately shortly after I started this project I changed employers and switched to using IntelliJ IDEA, so I stopped working on this.
Still, I occasionally get bug reports raised against it, and the Eclipse editor team approached me about adopting it and the project continues to accumulate stars on Github, so all things considered I’d say this project has been more successful than it had any right to be!
Snapsy, a scan-to-PDF app (C#)
I wanted to stop keeping paper around, which meant scanning existing paper into PDF format. There didn’t seem to be a good option for doing this in bulk with consumer scanners except a buggy & unmaintained open source project called NAPS, so I forked that project as Snapsy, added some features I wanted, and fixed a whole bunch of bugs in it. I used it for a year or two until I found a better workflow, at which point I deprecated the project.
Calamity, a client-oriented bug tracker (Python/Django)
To keep on top of issues and requests raised by customers of my Golf Booking SaaS business (and after getting fed up with the inadequacies of free solutions like Traq and Redmine), I built a bug tracking site. It was functionally very similar to early incarnations of Github issues, except specialized to make it easier for each of my customers to get a view of issues and activity relevant to them.
I improved and maintained this for about 3-4 years before eventually migrating everything to Trello in the name of simplifying the amount of services that I was running; at that time, several business concerns which had made it more compelling to roll my own solution originally had also dissolved, making deprecation of this a win all round.
Golf Booking SaaS (Python/Django)
As a paid side-project-turned-micro-SaaS-business, I built (and still maintain) a Golf Booking site & CMS for several Australian golf associations. I wrote these in Python using Django, and later added in a dose of managing WordPress, and everything is fronted by Nginx. The system has been rock solid for 8+ years in terms of functionality and reliability, even though my visual design work from ~2010 has some… room for improvement.
I also put in a lot of work to fully automate the bootstrapping and deployment of code for new VPSes using Ansible (containerizing wasn’t a viable option yet). Security is locked down, all configuration is stored as code, local and offsite encrypted backups are fully configured (you can see some of my Tarsnap tooling), and while it was probably overkill at the time as compared to maintaining a manual checklist, it taught me a great deal about Linux system administration, Ansible (which very importantly, helped me automate my Factorio server several years later!), scripting and has significantly eased my maintenance burden since then.
Automated SMS sender (C#)
I built a small application in C# which sent a different SMS message every day to a specific phone number by integrating with an ISP’s SOAP API. Technically straightforward in hindsight, but it was the first time I actually did something vaguely useful with an external API, and my first time building anything in C#.
Missile Command clone (Lua/Love2D)
A friend and I had a brief stint of collaborating on making a 2-player version of Atari’s classic Missile Command game in Lua, using the Love2D framework. We never actually shipped it (didn’t help that we were learning Lua at the same time!), but I enjoyed implementing bezier curves to make missiles smoothly travel from their silos to their targets :)
Low-end Webdev (PHP)
I’d been dabbling in HTML web development ever since I discovered Geocities (RIP), but it took me a while to realize you could execute arbitrary code on the backend too, using a magical thing called PHP. I promptly rebuilt my own personal website using it, then quickly thereafter landed a side-gig to build a website for an alternative modalities health clinic.
Early desktop apps (Visual Basic 6)
Some time during high school I acquired a copy of Visual Basic 6, which unlocked a whole new world for me. I had no idea what I was doing, but suddenly I could throw together forms to make desktop applications and I learned about calling Windows APIs and network programming using sockets too!
Suddenly there seemed to be no limit to what I could build, so I wrote a whole bunch of things:
- An app to track several characteristics for all my video games (e.g. CD locations, ratings, multiplayer capabilities), using a Microsoft Access database file as the data backend.
- A Windows system-tray-based application launcher.
- A clipboard manager for Windows.
- Several IRC bots, mostly for useless purposes.
- A UDP-broadcast-based network chat tool (together with a friend).
- A wildly incomplete Multi-User Dungeon, with just the network layer, room layout, basic navigation & combat implemented.
- A computer remote administration tool with screen grabs, mouse control, keyboard control, ability to disguise itself convincingly by replacing the Windows task manager, file browsing/transfer, execution of arbitrary code, keylogging.. okay, it was basically a virus in all but name, but in my defense, I only ever used it to keep tabs on and occaisionally annoy my younger brothers.
Early gamedev (Blitz Basic & Blitz 3D)
Aside from a brief flirtation with C wherein I got hung up on understanding pointers, the first language I coded anything serious in was a demo copy of Blitz Basic, a game-development oriented BASIC shipped on a CD of some computer magazine. Eventually during high school I got my hands on its successor Blitz 3D, and started making some 3D games.
I didn’t know about version control, so most of the sources have been lost, but some things I made were:
- A 2-player 3D car-driving sandbox, with no real objective other than doing flips and making patterns through the air via light particle sprites as you drove up implausibly steep heightmapped terrain.
- A partial clone of the 2D caveflyer Jetmen Revival, complete with spaceship physics & collision detection, and destructible terrain/particle/water physics, similar to falling sand games that came later.
- A 2D ragdoll physics sandbox based on a paper I read about verlet integration. Eventually I added a construction mode & breakable constraints, morphing it somewhat into a bridge-builder-esque sandbox.
- A 3-player 2D stick figure tag game, with stick figures painstakingly animated by hand.
- Countless number guessing games, silly screensavers, flashing desktops, snake/Tron/space-invaders/breakout clones, etc.