Building a Logger for a Custom Particle System Generator

In my previous Debugging class, my professor posed a rather interesting question: how would you build a logger? What constitutes a good logger? What should a logger display? As with most things in life, it depends. However it still is something to think about. The question really got me thinking, more so because I have been working on my own special effects program that creates and updates particle systems. If I were to create a debugging logger for a program designed for said program, how would I do it and what would it include?

My experience with debugging loggers

During my first and second year of making games at UOIT, the only logger I became familiar with was the Visual Studios error log and console. The error log allowed me to see blatant errors and warnings, while helpful, can only go so far. The console was useful for using cout to display fail/success of loading, if something couldn't be found, and more. The console proved used but was still very limited.

During third year, we used an engine which utilized a logger, but lacked a console. It would print relevant information to a text file and not overwrite information from previous program executions. The logger had some useful functions, as I would print out relevant game information. While useful, I felt it lacking, and I knew there could be more to it to help speed the development process.

How I would build my own

When I would make a logger, it would have two primary components: a printed element and a visual element. The printed element would print information to either a text file, console, or both. This could range from whether or not a third part library initialized, if an asset loaded, if some arbitrary class initialized, or other wise. The possibilities are really limitless.

When creating a custom particle system generator, I need to consider what my program would be doing. For example:

• Number of different particle systems being updated
• If they successfully initialized or not and related information such as:
• Number of particles
• If assets for those particles loaded or not
• The name of each particle system
• If I push a button on the GUI
• If something goes wrong, how long in the simulation was it?
• A generic error function that prints as much who/what/where/why/when info as possible

Printing to a file for debugging

The reason why I'd want to know all of these things is so, if there ever is some kind of problem right away, I can open up my log and have all of the relevant facts to analyze. When a problem occurs, I can save a copy of that text file, repeat the problem, then analyze my logger to try and determine the issue. Here is a mock class I wrote to visualize my logger class.

Obviously this would not be the final product but it might use a lot of the functionality provided. What is important to note is the inclusion of the function error. Error's purpose would be to trigger upon something failing utterly, and subsequently print the who, what, where, why, and when, if possible. It can get the who, where, and what by knowing what part of the program it failed (for example, when loading an asset), when, simply being the time. Knowing why could be as simple as, "it failed to load a file" to something not being properly defined. However, the program may now always know exactly why, so sometimes we may have to take that part with a grain of salt, depending.

Printing to the screen for debugging

The visual element would display live data on the screen or to a console. This data could range from player position, CPU usage, FPS, number of entities within a vicinity of the player, game time, and more. This way, the debugging has the most critical data on screen.

A good example of a visual debugger in a game

Unfortunately a console is limited to pretty much just text. While useful, it overlaps the duty of a text file. I would still use it to print initialize data, but I wouldn't invest to much time in it. When making a visual element for my particle system generator logger, I would want this information displayed live:

• Number of particle systems, and how many are active
• Number of particles. 
• Total 
• How many are being updated all at once?
• How many are being updated by a particular system?
• Inactive particles
• CPU usage
• Simulation time
• Display the information of a singled out particle system.

Here is an updated class:

While it is not much of a change, the display function is going to have a huge amount of responsibilities. It will take in two pointers, particle manager and the active particle system. Using that, it would print relevant information to the foreground, so that it doesn't get blocked by anything on screen. Also, a simple key should be used to turn it on and off. I like to use the '5' key for debugging purposes. 

I would make it so that it prints any encompassing information, such as total particle systems, number of active, etc, to the top left. Below it, would be the singled out particle system data, and to the right, system data such as RAM and FPS.

Example mock-up live display debugging screen

Conclusion

One of the more important things I've learned from my game dev experience is that knowing how to debug is absolutely critical. I also know it can be a painful experience. When making a logger, I would try to make it in such a way that it would make fact analysis and information gathering easier. While what I've displayed is only a mock up, it gives across the right idea of what to think about when making a logger.

Thank you for reading! Hope you learned something.

Debugging by Thinking and Professor Solomon

Previously, I wrote a blog describing my old debugging technique juxtaposed to how book Debugging my Thinking discusses a more Sherlock Holmes way of approach bug finding. However, the textbook describes more then one approach to thinking about debugging then just Sherlock Holmes. The text describes the methods of Professor Solomon's 12 steps to finding lost objects and how you can use the same thought process to locate bugs in code. Today I'll be analyzing those 12 steps, how they can apply to game development programming, and any thoughts I have about them. 

When applying Professor Solomon to debugging, you have to try to make an analogy between lost objects, and software defects. The lost object is an action of lack of action occurring in some unknown location. The visibility of that object is understanding how and where that piece of code causes the symptom. 

The Methods of Professor Solomon

• Don’t look for it
• It’s not lost, you are.
• Remember the three c’s.
• It’s where it’s supposed to be.
• Look for domestic drift.
• You’re looking right at it.
• The camouflage effect
• Think back
• Look once, look well
• The eureka zone
• Tail thyself
• It wasn't you.

Don't look for it

Don't start looking until you have an idea where to look. This is debugging by thinking! Not by hacking code and hoping stuff will work. Don't look at source code once you find a defect and don't look at output once you found or received evidence of the defect. Instead:

• Make a list of criteria that will qualify or disqualify sections of code.
• Make a list of similar defects you have seen in the past.
• Make a list of root causes and try to narrow down possibilities.

Once you have your list, start narrowing down possibilities. Remember, it is about knowing how to look, not where to look.

It's not lost, you are

Bugs are accidental and a behavior that is as yet unexplained. When looking for bugs, it is important you use a logical system for narrowing down your possibilities and finding it. It is important that you use a system for searching. If not, you will waste a lot of time, effort, and sanity, looking for it. Use a system!

Remember the three C's!

To find a bug, you have to be in the right mindset. There have been to many bugs I have chased and chased when frustrated, only to end up wasting my time. The three c's are:

• Comfort
• Calm
• Confident

It is important to be in a healthy state of mind and body, especially when tackling those bugs that has taken up days of a teams time. Drink plenty of water, breathing exercising, listen to calming music, take short breaks, proper lighting, etc. When you are physically and mentally capable of conquering bugs in code, chances are you will find them faster.

It's where it's supposed to be

When a piece of code is causing a bug, that bug isn't moving anywhere. It is exactly where it is supposed to be. But how can we find it? Using deduction, we can find where the bug IS NOT located. Try to narrow down locations where you can know, for certain, that the bug is not located, you can save yourself a lot of searching and stress. Remember, the bug is exactly where it is supposed to be, you can use that logic to narrow down where it isn't, and find it.

Look for domestic drift

Most bugs are normally found in the place you last modified. However, that does not mean you should right away look there. Use deduction to determine the root of your problem. When there is a defect, it will generally have a root point where it defects, then it will cause a trickle down effect where it causes a sequence of problems. Look for those kind of drifts. As a tip, try making a flow chart that illustrates the flow of data in your suspected area to try and narrow down what might be causing the issue. 

You're looking right at it.

During third year, I had a bug where items were not being properly parented. I had skimmed my code a hundred times but didn't see the problem. When trying to locate a bug, remember, you may be staring right at it and not realize that is the bug! Before looking for bugs, try explaining the bug to a friend or co-worker. Write down, logically, what is supposed to happen step by step and eliminate impossibilities. Narrow down your problem to a section of code and look viciously. 

The camouflage effect

I do not believe I have personally experienced this problem, but it is something to beware of. If you are absolutely certain you have narrowed down a problem to a key area, but cannot determine what may be causing the problem, consider that there are other factors that can place a problem out of view. For example:

• Preprocessor macros
• Procedure Calls
• Exception handling
• Complex language constructs (C++)

Think back

This section only applies to code you created. Try to think about what you did in terms of changes, and how that would alter the flow of logic you were intending to create. What is best for this section is to create versions of your code frequently so you always have a backup to look at. Also, commenting your code frequently, and providing large blocks of comments that describe changes, helps. It can give you a good idea of what you intended to do which can subsequently help you narrow down a list of problems you should create. 

Look once, look well

I wish I followed this one better. I mentioned earlier I had a problem with parenting in a game I was making in third year. I looked in so many different cpp files that it was giving me headaches. What was worse, was that I looked at the same spot several times, even when I thought it wasn't the problem area. When it comes to debugging, look once, look well. 

At each potential problem area, use all relevant tools to help find the bug. As mentioned before, also keep different versions so you can always go back and look. However you have to be wary when taking this advice. Make sure that, when you do look, you either confirm or eliminate the possibility of the problem being in that location. If you simply do not find the bug, check your other potential problem areas. If you know the problem is in that area, look again and try to understand the logic of what the program is supposed to do.

The Eureka Zone

More often then not, bugs develop where they should, others occur in the immediate vicinity. The textbook describes two types of displacement, physical and temporally. Temporally displaced problems are described as:

• Recently modified variables
• Recently executed statements
• Procedures closest to the call stack

Physically displaced problems often have to do with pointers. This is what the textbook suggests:

• Suspect variables in the same heterogeneous storage construct as the variable in which the problem became visible
• Suspect references that are on the call stack at the same time
• Suspect references to storage allocated on the heap at about the same time as the variable in question. 

Tail Thyself

Certainly the fanciest sounding tip and perhaps one of the most important. The concept is simple, tail exactly what you did and how you got to the point you are at. Keep versions, code comments, flow charts for logic, even if you have a written description of what you wanted to achieve, anything that can make you follow in your footsteps to help you narrow the problem down. Using revision software is critical for this step.

It wasn't you

After your herculean effort to try and figure out your problem, it may be reasonable to assume that you're not the cause of the defect. This one only applies if you're working with other people, or some other software development kit. With this problem, ask questions and do some looking around to see who or how the problem came about. If it is someone else in a group, then simply ask them. But make sure to have at least tried to narrow down the problem and create a bug report that is useful to that person. If you're using a SDK, then create a test case to demonstrate the problem and ask a thorough question. No matter how dumb your problem may be, never be afraid to ask.

Conclusion

When debugging, it is important to look at bugs as lost objects that you have to find. Using simple logic and a systematic way of finding bugs, you can reduce your stress, increase your productivity, and simply have more fun coding. I would say the most important take away from Professor Solomon's tips are to have a system for searching and remaining calm, comfortable, and confident. Thank you for reading, I hope you enjoyed my latest post!

Debugging by Thinking and Sherlock Holmes

Going into the class Debugging Techniques I thought it would be  about using tools, understanding compilers, and learning about some debugging techniques. What I did not expect is that it would be more about thought process then raw technical skills. After reading the first four chapters of the textbook, I already have found new ways to enhance my approach to bug handling. Utilizing the thought processes of a fictional character such as Sherlock Holmes can allow a programmer to exceed their ability to debug code, just by thinking about the problem effectively. Today I'm going to be looking at some bad practices that the textbook points out, my bad (and good) practices, and how the textbook has already shown me ways of improving my already existing methodology.

The class textbook, Debugging by Thinking, defines three rudimentary ways of debugging code that novices generally do.

Debugging by editing: 

• Making some changes and executing the program again.

Debugging by interacting: 

• Using a debugger to observe some aspect of the programs behavior before making a change.

Debugging by repeating:

• Programmer has a solution to the problem, but does not fully understand the assumptions or limitations of the solution. They will apply the action set to every problem, but it is more of a bag of tricks then an actual understanding of the real problem at hand.

Not only am I guilty of every single one of these, I can already see how they can be a big problem. While I myself have developed my own ways of debugging my code, not only do I not always follow them, there is room for improvement. Here is what I generally do:

1. Observe a problem
2. Try to repeat the problem
3. Look at where I think the code is (place break points)
4. Try to determine the cause
5. Make a change
6. Execute program!

While I thought this process might have some positives,upon further analysis it has heaps of problems and I don't follow that methodology every time. I sometimes debug by editing, or simply just go into my old bag of tricks and 'fix' a problem without even understanding how or why it fixes it. Also, I tend to not write down any notes, get frustrated, very little induction or deduction, and I check the same spot over and over, even if I didn't see a problem there. I could go on more about the poor decisions I make, but rather I will say how I can fix them by using Sherlock Holmes logic.

The methods of Sherlock Holmes

Sherlock has a list of methods that one can use to help debug their code. 

• Use cross-disciplinary knowledge
• Focus on facts
• Gather facts before hypothesizing
• Pay attention to unusual details
• State the facts to someone else
• Start by observing
• Don’t guess
• Exclude alternative explanations
• Reason in both directions
• Watch for red herrings

Applying Holmes to debugging

You will want to determine the contributing causes, how and when did the code do its unexpected behavior? Narrow down code to find the code that had the means and opportunity to cause an observable failure and then focus on that code segment. This way, you have isolated the potential problem and can go from there.

Focus on Facts & Gather facts before hypothesizing:

You want your events to be reproducible so you can gather as many facts about the potential bug as possible. With that, you also want to try and find something that has access to the means and opportunity to cause the defect. While repeating, start logging what it could and could not be, so you can start developing a bug report. 

When I would debug, I rarely ever focused on the facts. Not only will I collect facts as I go, I'll use those to form a hypothesis from them. I'll be using the scientific method for trying to solve my problems. This involves:

1. Stating the problem
2. Forming a hypothesis
3. Observe and experiment
4. Interpret the data
5. Draw conclusions

Once I solve said problem, I'll make my own solution method in the event I run into that bug again.

Pay Attention to Unusual Details

When something goes awry, it is always wise to check what the common behavior should be. This allows us to look at a problem knowing what it should be doing and what to look out for. This allows us to gather facts to build towards our hypothesis, or to help confirm it. When debugging, we need to collect as much data as possible to help us understand the crux of the issue.

State the facts to someone else

This is something I do often. I will tell someone what the problem and what I have tried. More often then not, I have a moment of clarity and solve the issue. Even if that person might not understand a word you're saying, it still helps to hear it out loud. Develop a viewpoint by answering these questions:

• What do I know for sure?
• What do I believe to be true, and not true?
• What do I not know?

Start by observing

Detecting and debugging differ to the extent to which we can create additional evidence. When we debug software, we can repeat the defective behavior and observe. This could be watching or recording the behavior, reading outputs from the program, or more. Consider from what view point to observe or how application performance analysis tools collect data. Performance information can be collected either when an event occurs or at fixed time intervals. When debugging, log when information either at events or fixed time intervals to understand the evolution of the segment you are analyzing. 

Don’t Guess

When discussing with a friend some examples of this, we used drinking from a lemon-aid plastic bottle as an example. My friend was drinking a clear liquid from a bottle of lemon-aid, but it wasn't lemon-aid. Even though one could safely assume it would be lemon-aid, upon further inspection you would notice it clearly is not lemon-aid, only water. Sometimes, even the most obvious of things is something you shouldn't guess.

When programming, I find guess work debugging not only costs time, but energy. The textbook states you shouldn't change something unless you fully understand its effect on the software. Hacking is an even worse idea because it's a poor way of solving legitimate problems. Even if the problem goes away, if you encounter it again, you won't really be fixing it and you'll rely on a bag tricks. 

By understanding the problem in its entirety, you get to understand the software so much more and how to approach that problem again, if you ever see it.

Eliminate impossible causes

The quote, "When you have excluded the impossible, whatever remains, however improbable, must be the truth" comes to mind when thinking about eliminating impossible causes.

When trying to narrow down a defect, try to eliminate as many possible causes and culprits as you can with a single observation. However, before stating something is impossible, make sure it is. Any evidence that may make it possible, may still make that potential segment a possible problem. For this, it is best to log what is and is not impossible. Use code comments, or a text file on the side. Anything to help gather facts and form your hypothesis. 

Exclude alternative explanations

When you formulate a hypothesis, consider if you already have evidence in hand that disqualifies the hypothesis. As mentioned before, this is easier if you keep a log of your observations and hypothesis.

One thing that I would often do is conduct experiment after experiment without fully updating or disqualifying my hypothesis. It takes less time to disqualify a hypothesis in this way than to perform yet another experiment. I never truly considered brainstorming a list of all the hypotheses that this experiment disqualifies, something I want to try. 

Reason in both directions

The bane of my debugging methodology! Very rarely would I take a sequence of events and it's result and try to deduce the problem. The textbook gives two logical processes of determining a problem.

• Deduction:
• The process of reasoning in which a conclusion follows necessarily from the stated premises, inference by reasoning from the general to the specific.
• Induction:
• The process of deriving general principles from particular facts of instances.

I've learned from this section that a programmer should try to look at an event as more then just some problem, but as a symptom of a defect. That symptom can then be diagnosed and be analysed for potential causes, and we can reason from the symptom what the source might be. 

Watch for red herrings

Sometimes when programming, I would encounter a problem and say, "nah it can't be that". Never did I ask, "could it be this, and if so, how?" Red herrings are a piece of information is so obvious, it distracts us because of its deceptively obvious nature. 

For example when working on my third year game development workshop game, I was encountering a problem where objects would form parent-child relationships. After trying to solve the problem for so long, and even saying to myself, "it can't be a parenting issue", I eventually found out the hard truth. It was so painfully obvious that it was a parenting issue that I overlooked the fact that it was. 

When you come across a fact, consider all of its interpretations. Keeping a debugging log can help speed up acquisition and reduce future debugging time. More often then not, it is the most obvious fact that eludes us the most frequently. 

Summary

Sherlock Holmes has an incredible thought process that is very applicable to programmers. So far, the textbook has provided a solid foundation for making me a thinking debugger and I plan on using the knowledge in it so far to put myself into that mindset. If I become some sort of super debugger because of this, I'm buying a deerstalker hat and a pipe for my investigating. Hope you enjoyed the read!