Debugging by Thinking and Sherlock Holmes

Going into the class Debugging Techniques I thought it would be  about using tools, understanding compilers, and learning about some debugging techniques. What I did not expect is that it would be more about thought process then raw technical skills. After reading the first four chapters of the textbook, I already have found new ways to enhance my approach to bug handling. Utilizing the thought processes of a fictional character such as Sherlock Holmes can allow a programmer to exceed their ability to debug code, just by thinking about the problem effectively. Today I'm going to be looking at some bad practices that the textbook points out, my bad (and good) practices, and how the textbook has already shown me ways of improving my already existing methodology.

The class textbook, Debugging by Thinking, defines three rudimentary ways of debugging code that novices generally do.

Debugging by editing: 

• Making some changes and executing the program again.

Debugging by interacting: 

• Using a debugger to observe some aspect of the programs behavior before making a change.

Debugging by repeating:

• Programmer has a solution to the problem, but does not fully understand the assumptions or limitations of the solution. They will apply the action set to every problem, but it is more of a bag of tricks then an actual understanding of the real problem at hand.

Not only am I guilty of every single one of these, I can already see how they can be a big problem. While I myself have developed my own ways of debugging my code, not only do I not always follow them, there is room for improvement. Here is what I generally do:

1. Observe a problem
2. Try to repeat the problem
3. Look at where I think the code is (place break points)
4. Try to determine the cause
5. Make a change
6. Execute program!

While I thought this process might have some positives,upon further analysis it has heaps of problems and I don't follow that methodology every time. I sometimes debug by editing, or simply just go into my old bag of tricks and 'fix' a problem without even understanding how or why it fixes it. Also, I tend to not write down any notes, get frustrated, very little induction or deduction, and I check the same spot over and over, even if I didn't see a problem there. I could go on more about the poor decisions I make, but rather I will say how I can fix them by using Sherlock Holmes logic.

The methods of Sherlock Holmes

Sherlock has a list of methods that one can use to help debug their code. 

• Use cross-disciplinary knowledge
• Focus on facts
• Gather facts before hypothesizing
• Pay attention to unusual details
• State the facts to someone else
• Start by observing
• Don’t guess
• Exclude alternative explanations
• Reason in both directions
• Watch for red herrings

Applying Holmes to debugging

You will want to determine the contributing causes, how and when did the code do its unexpected behavior? Narrow down code to find the code that had the means and opportunity to cause an observable failure and then focus on that code segment. This way, you have isolated the potential problem and can go from there.

Focus on Facts & Gather facts before hypothesizing:

You want your events to be reproducible so you can gather as many facts about the potential bug as possible. With that, you also want to try and find something that has access to the means and opportunity to cause the defect. While repeating, start logging what it could and could not be, so you can start developing a bug report. 

When I would debug, I rarely ever focused on the facts. Not only will I collect facts as I go, I'll use those to form a hypothesis from them. I'll be using the scientific method for trying to solve my problems. This involves:

1. Stating the problem
2. Forming a hypothesis
3. Observe and experiment
4. Interpret the data
5. Draw conclusions

Once I solve said problem, I'll make my own solution method in the event I run into that bug again.

Pay Attention to Unusual Details

When something goes awry, it is always wise to check what the common behavior should be. This allows us to look at a problem knowing what it should be doing and what to look out for. This allows us to gather facts to build towards our hypothesis, or to help confirm it. When debugging, we need to collect as much data as possible to help us understand the crux of the issue.

State the facts to someone else

This is something I do often. I will tell someone what the problem and what I have tried. More often then not, I have a moment of clarity and solve the issue. Even if that person might not understand a word you're saying, it still helps to hear it out loud. Develop a viewpoint by answering these questions:

• What do I know for sure?
• What do I believe to be true, and not true?
• What do I not know?

Start by observing

Detecting and debugging differ to the extent to which we can create additional evidence. When we debug software, we can repeat the defective behavior and observe. This could be watching or recording the behavior, reading outputs from the program, or more. Consider from what view point to observe or how application performance analysis tools collect data. Performance information can be collected either when an event occurs or at fixed time intervals. When debugging, log when information either at events or fixed time intervals to understand the evolution of the segment you are analyzing. 

Don’t Guess

When discussing with a friend some examples of this, we used drinking from a lemon-aid plastic bottle as an example. My friend was drinking a clear liquid from a bottle of lemon-aid, but it wasn't lemon-aid. Even though one could safely assume it would be lemon-aid, upon further inspection you would notice it clearly is not lemon-aid, only water. Sometimes, even the most obvious of things is something you shouldn't guess.

When programming, I find guess work debugging not only costs time, but energy. The textbook states you shouldn't change something unless you fully understand its effect on the software. Hacking is an even worse idea because it's a poor way of solving legitimate problems. Even if the problem goes away, if you encounter it again, you won't really be fixing it and you'll rely on a bag tricks. 

By understanding the problem in its entirety, you get to understand the software so much more and how to approach that problem again, if you ever see it.

Eliminate impossible causes

The quote, "When you have excluded the impossible, whatever remains, however improbable, must be the truth" comes to mind when thinking about eliminating impossible causes.

When trying to narrow down a defect, try to eliminate as many possible causes and culprits as you can with a single observation. However, before stating something is impossible, make sure it is. Any evidence that may make it possible, may still make that potential segment a possible problem. For this, it is best to log what is and is not impossible. Use code comments, or a text file on the side. Anything to help gather facts and form your hypothesis. 

Exclude alternative explanations

When you formulate a hypothesis, consider if you already have evidence in hand that disqualifies the hypothesis. As mentioned before, this is easier if you keep a log of your observations and hypothesis.

One thing that I would often do is conduct experiment after experiment without fully updating or disqualifying my hypothesis. It takes less time to disqualify a hypothesis in this way than to perform yet another experiment. I never truly considered brainstorming a list of all the hypotheses that this experiment disqualifies, something I want to try. 

Reason in both directions

The bane of my debugging methodology! Very rarely would I take a sequence of events and it's result and try to deduce the problem. The textbook gives two logical processes of determining a problem.

• Deduction:
• The process of reasoning in which a conclusion follows necessarily from the stated premises, inference by reasoning from the general to the specific.
• Induction:
• The process of deriving general principles from particular facts of instances.

I've learned from this section that a programmer should try to look at an event as more then just some problem, but as a symptom of a defect. That symptom can then be diagnosed and be analysed for potential causes, and we can reason from the symptom what the source might be. 

Watch for red herrings

Sometimes when programming, I would encounter a problem and say, "nah it can't be that". Never did I ask, "could it be this, and if so, how?" Red herrings are a piece of information is so obvious, it distracts us because of its deceptively obvious nature. 

For example when working on my third year game development workshop game, I was encountering a problem where objects would form parent-child relationships. After trying to solve the problem for so long, and even saying to myself, "it can't be a parenting issue", I eventually found out the hard truth. It was so painfully obvious that it was a parenting issue that I overlooked the fact that it was. 

When you come across a fact, consider all of its interpretations. Keeping a debugging log can help speed up acquisition and reduce future debugging time. More often then not, it is the most obvious fact that eludes us the most frequently. 

Summary

Sherlock Holmes has an incredible thought process that is very applicable to programmers. So far, the textbook has provided a solid foundation for making me a thinking debugger and I plan on using the knowledge in it so far to put myself into that mindset. If I become some sort of super debugger because of this, I'm buying a deerstalker hat and a pipe for my investigating. Hope you enjoyed the read!