My only skill is debugging. I have worked as a programmer — software engineer, software developer, or whatever you want to call it — for many years.
I possess only this one skill. It is the only skill I have required.
I: Building Blocks of Debugging
In 2007 I was unemployed and living with a friend in my college hometown. The room I rented was shared with my roommate’s pet rabbit. I had an old suitcase with some clothes and toiletries, some books, and a cheap laptop.
I had a college degree, but I did not have a plan. The more salient issue is that I did not have any money. I had worked an internship at a bank and decided it wasn’t for me. Since I couldn’t decide what I wanted to study in graduate school, I needed to get a job.
I knew I was going to be a programmer of some sort, and I knew I wanted to do something with the internet, but I didn’t know what. I had learned web development and designed some websites in high school for fun. I liked the internet. I liked making things.
I liked the logical structure and rigor of math and physics. But I also enjoyed reading and writing. I found programming an interesting combination of mathematical thinking and the kind of creative expression found in poetry.
I set out to make a living from these building blocks, with absolutely no idea where it would take me.
II: Discovering the Skill of Debugging
I got hired by a small local company in central Illinois developing a CMS. I had to google “CMS” when I started. I had no idea how a CMS worked, but I had seen some PHP code before, and I had written a bunch of Perl in high school and college.
The job itself was unglamorous. I was given an old desktop PC and a monitor than took 20 minutes to turn on every morning.
I found out pretty quickly that I needed to get my workspace setup so that I could “fiddle around” with the code. All I could really do with the PHP environment was change something and reload the page.
As rudimentary as this setup was, it was enough for a simple test.
Load the page with state A. Observe. Load the page with state B. Observe. Note the differences, if any.
Gradually, I was able to make sense of what each piece of the application was doing. With that bit of knowledge, I could then start fixing problems in the PHP code. The more of these experiments I ran, the more I understood.
It wasn’t my dream job, but I made a bit of money, I paid rent, and I bought groceries. I was a web developer! Mission accomplished.
III: Debugging As A Skill
I am writing this thirteen years later. My work is different now. I have a nice computer that doesn’t take 20 minutes to turn on. I have more experience. But otherwise things haven’t changed.
I do not possess extraordinary talent. I am not an expert in obscure algorithms, nor do I possess any other highly specialized qualifications. It is often suggested that specialized skills are necessary to prosper in the modern economy. Yet my only skill is a combination of basic logical reasoning, note taking, and stubborn persistence.
In other words, I figure stuff out for a living.
The process begins when I run into a problem. By carefully reasoning through the problem, gathering evidence, making hypotheses about what might be going on, and trying to disprove these hypotheses, I arrive at otherwise inaccessible truths about how a system works.
This collection of truths allow me to start solving the problem. The sum total of these investigations gets processed in my brain into a model of how the system works, and I compare that acquired model against other systems I’ve seen. I can then make some educated guesses about what to investigate next.
In software parlance, we call this process debugging.
Debugging is simply the process of figuring stuff out. Often the motivation for debugging is to fix a problem (a “bug”) in software. The process of building new code often involves debugging, too, since the new things we put together never work quite like we envisioned.
Most programming jobs do not require an extensive background in theoretical computer science or any specialized expertise. It is far more important that someone working in this field be good at logical reasoning and have a knack for expression.
Another crucial aspect of the job is tolerance for petty frustration. Debugging is tiresome and requires a laser-like focus on tiny, fiddly details. I often feel more like a forensics expert examining an exhumed corpse than like an architect designing a beautiful majestic cathedral.
IV: Debug Loops
When you need to debug something, you’ve got to get into a good tight debug loop.
What is a debug loop? I define it as the circular process of forming a question, asking the question, getting an answer, and deciding what question to ask next. This question->answer->question cycle is the loop. We want the loop to be tight so we can ask lots of questions, and get lots of answers.
Often that answer comes from a logging statement, a metric value, or a thread interrupt pointing us to a line of code.
The technology involved isn’t so important, we just want that loop to be as tight as possible. If we have to wait hours or days to get an answer, it’s going to be really hard to debug, since our brains will get distracted with something else while we wait.
Step one of programming is setting up this debug loop to set ourselves up for success. Sometimes that means we have to go do some other work to make our debug loop shorter.
V: Debugging Is About Understanding People
It is a misconception that programmers need to understand computers. Computers are supremely rational and do not take much work to understand. It is humans that are hard to understand.
All computer systems are designed by humans. Even code generated by computers is produced by programs written by humans, so it all comes back to human fallibility in the end.
Computer systems are obnoxiously literal. Programming languages translate the programmer’s written instructions into a form the machine can understand. But they are woefully useless at interpreting a programmer’s intent, and are very unlikely to improve at this task anytime in the next century.
When we debug systems in the real world, we are not looking for problems within the computer. We are actually looking for mistakes that a human made.
Very often these mistakes are not the volitional act of one bumbling programmer, but rather the accumulated mistakes of many people working under pressure who were themselves working around a previous mistake.
So it helps to have a Sherlock Holmes attitude when debugging. We are not really trying to figure out why a machine isn’t working, but rather trying to figure out what confluence of human decisions have led to the program doing what it does.
VI: Primitive Debugging
Debugging is not a new technique, and it does not require knowledge of any technology to perform. It is just a natural application of logical reasoning to a computer program.
In fact, all you need for debugging is a tool invented about 2,500 years ago: the syllogism.
A syllogism is a simple logical inference made from two statements. In short:
if a then b
if b then c
therefore: if a then c
It’s also important to know the logical inverse of this operation:
if a then b
if b then c
therefore: if NOT c then NOT a
…which is really the same formula but with a NOT around it.
When we discover some truth, we can plug it into the inputs for this formula, and generate new outputs. That is basically all we’re doing when we debug a program.
VII: Debugging Is Enough
Credentials and specializations may confer the illusion of skill. But the real skill in any programming or programming-adjacent job is debugging. I contend that this skill alone is sufficient for the majority of programming.
This is not to say that everyone who understands reasoning can get any programming job or work on any project. Anyone who understands reasoning can debug any system, but it may take longer for someone without knowledge compared to someone with knowledge who is already further along in the debug loop sequence. Knowledge effectively lets us “jump ahead” to ask more specific questions and solve the problem sooner.
There is a difference between knowledge and skill. Knowledge provides the context for performing a task. It is skill that actually performs the work.
The practice of debugging often produces knowledge. After we have debugged a system, we understand it on a level that would be harder to acquire through documentation or books. The knowledge produced by debugging is a kind of accumulated knowledge, earned by reasoning through the system’s behavior.
Yet the skill of debugging remains the crucial ingredient, because it is both the cognitive tool used to perform the work and the process for gaining knowledge.
Gradually, over time, I have added to my set of accumulated knowledge until I am familiar with programming models and concepts. Yet this experience has not helped me to debug more effectively. I am still about as good at debugging as I have ever been.
This meager skill has sustained me now for some thirteen years and counting. Perhaps in another thirteen years I will have acquired some additional skill.