The Importance of Bug Reports

Fixing your own bugs is all fine and dandy, but what about fixing other peoples problems? One of the quintessential components of debugging is knowing how to write a proper bug report. We were shown how to write out own bug reports in class, but in today's post I want to strain the importance of the bug reports including key sections to make them as clear and concise as possible.

The way I view bug reports is this: if you spend an extra 5 minutes making one, you may save someone an hour of their time. Detailing the report with as much evidence as you can find using concise text and images may take longer, but will help whomever views the report greatly. Different key sections would include:

• Technical details (OS, compiler, third-party libraries, etc)
• Summary of the program or section of the program the bug exists in.
• Severity
• Summary of the problem.
• How to create the process.
• Past attempts and results.
• Any debug info (call stack, debugging log files, console output)

I will be using a program from class I created and purposely broke for homework purposes. I will build a bug report out of it for you to follow along too.

Technical Details

Rather self-explanatory, technical details is a list that shows your hardware and software. This includes OS, third-party libraries, any API's or SDK's, If you're program runs on a PC, include your specs if you truly feel its necessary. For AAA PC games, specs are certainly useful, but for a small project between buddies in University may not be as critical.

Summary of the program or section.

Rather self explanatory, this section details what your program should do. Feel free to summarize your program as a whole if it is small enough (for example, a basic addition/subtraction program), or describe what particular section is breaking (when you try to grab loot after defeating a particular boss). If the program is small enough, you can include summary of the program and summary of the problem in the same section, which is what I do in this case.

Severity

How severe is the bug in relations to the final release of the game. My professor gave us three levels of severity: A, B, and C. A means the product cannot be set to release (for example, crashes on start up). B is still severe but the game is playable (for example, lag spike when loading level 3). Several B bugs could be as serious as an A bug. C level bugs are small but don't really effect the grand scheme of things. They are still important to squash, but won't be a big hindrance on your final project (for example, if you unplug the controller 10 times really fast, the game crashes).

Summary of the problem 

The summary of the problem should be a quick, clear, and concise description of the expected behavior and the deviated behavior. A bad example of a summary would be, "it breaks when it compiles". Receiving this on bug reports makes me immediately send it back because nothing is really stated. What is the error? Is it a buffer overrun or linker errors? Do the textures flicker, and if so, how frequently and at what position? Summarizing the problem in a clear manner is critical to getting other people to understand it.

How to recreate the bug

Arguably the most important section, knowing how to recreate the bug is of utmost critical importance. This section I stress heavily and I mean HEAVILY taking the extra time to provide as much detail as possible into recreating the bug. Make a step by step process as if a 10 year old was trying to create it. Provide images, even a video if you feel it necessary, just make sure the person on the other side can recreate your bug the first time. By spending an extra five or ten minutes making sure you provide as much detail as possible can save someone else hours.

Past Attempts and Results

While not necessary for all bug reports, this section can come in handy if your team had already tried some different tests and could not determine the problem. If you are going to include past attempts and results, then treat it like a "How to recreate the bug" section and provide all the necessary details into what you did and what the results were. This can be useful in smaller group projects when your partner programmer details different things they tried, the results, and brings it to you saying they are stuck.

Note: I did not include this section in my built bug report.

Debug Info

Providing debug info can make or break finding a bug. Debug info can include the call stack, any program specific log files, console outputs, debug symbols, etc. This information acts as a breadcrumb trail that can help lead programmers on the right track to solving a bug. You may want to include this in a separate text file. Not all bugs need you to provide all of this information, but it can really go a long way to helping solving the problem.

My debugging section is rather lackluster as it could include more information, but it gets the point across that the console is not displaying any issues and that we are including debugging symbols.

Summary

That concludes this weeks post on bug reports. I hope you learned something new about the importance of bug reports. 

How not to debug, Part II

In my last post, I gave some general hints and tips on how not to set yourself up for debugging. This would include practicing ergonomics, getting into the right mindset, proper coding practices, documentation, commenting code, and more. Today, I'll be taking all of those and actually debugging some real problems I encountered over the years, and how concepts from class lectures, the textbook, and my previous blog all apply. Without further ado, let's get started.

Example problem

Last month I was working on an assignment for my artificial intelligence class. In this assignment, the user was placed in a room filled with 20 doors and each door was assigned three properties:

1. Cold or hot.
2. Noisy or quiet. 
3. Safe or unsafe.

Each door would be assigned one from each pair, so a door could be cold, noisy, and safe, or hot, quiet, and safe, and so on. I ran into two significant bugs while working on this project.

1. Noisy doors should play their sound when within close proximity. Regardless of proximity, the doors were playing their sound.
2. Doors were not being assigned the variables correctly. 

I will analyze the number 1 first. So, as mentioned, each door should be noisy when close. However, regardless of proximity, the sound plays. This is my current scene:

Noisy doors problem

Four doors are cold, one is hot. They are all defaulted to safe and only the hot one is supposed to be noisy. From where I stand in the scene currently, a lion roar sound plays. This should only play when directly in front of the door. Because the title of today's post is how not to debug, let's go over some ways not to debug this particular problem

What not to do: Start changing code.

While it may seem very, very attractive to just start playing with values, checking initialization, moving stuff around, etc, this is not how you should start out this problem. When you begin immediately start changing code without looking into how or why it defects, you create a habit where you do not think logically about your problem. This in turn can waste your time and create artificial stress. You could easily play with little segments of code for hours before actually figuring out the problem, to which you may not understand how you fixed it, or accidentally creating even more problems.

How to do it instead: What should I do instead?

Recreate the problem, or at least try too. Try to remember exactly how you got to the problem and recreate it. And when you do recreate it, either write it down or remember it, because this will help you develop a hypothesis later.

After you recreate the problem in code, you can do one of two things, and I believe both are valid depending on certain circumstances: Google (research) the problem or use all tools available (breakpoints, call stack, any other tools) to try and narrow down the problem so you can at least get an idea of the problem. I only advocate for researching when, based on the defect and how you made it, could be fixed by a quick search into documentation or Google. 

Actually Solving it

So instead of just changing variables, I look into the problem. I place break points at initialization, the play function, and at other points of interest, view the call stack, write a few lines of code to cout if some information is initialized correctly, a practice I should have done before, etc. 

I see that the sound is playing from exactly where it should be, the listener is exactly where it should be, and the sound has the correct set up in terms of how the sound resonates, falls off, and volume. So if it not an apparent problem with the code I wrote itself, it could be a problem elsewhere, such as:

• SFML
• I screwed up on the audio file export
• Could still be a problem with my code, but it's looking very unlikely

If it is a problem with SFML, then I can find it in two different locations, Google and documentation. I resort to Google first and after scrolling and scanning through a few different links, I find that SFML had bugs for stereo sound, not mono. My audio file was exported as stereo. After a quick export of the file in Mono, load it in, test, bam it works. While the way I fixed it was valid, another way would have been to load up the documentation on SFML's sound class and simply skim it. It says it blatantly in the document. 

The problem was relatively simple, but a lot of programmers can fall into the trap of not debugging correctly by looking for a problem that they don't understanding.

Doors not being assigned truth table correctly

So I had spent hours into the night coding the assignment of variables for the doors. Essentially the, the doors are assigned variables based on a probability. See the chart below.

Hot Noisy       Safe Door           Percentage of Doors
Y Y              Y                             0.05
Y Y              N                             0.10
Y N              Y                             0.03
Y N              N                             0.21
N Y              Y                             0.06
N Y              N                             0.11
N N              Y                             0.40

N N              N                             0.04

The problem I encountered was that the doors were not randomizing properly. A quick run down of the program: upon initialization, it creates the room and loads the door property text file. For each property, it uses the percentage value to determine what the door should be. I have two std::vectors, one contains ID values for each set of properties (values 1,2,3, etc), and another vector the probabilities (represented by an int value, so I multiple the percentages by 100).

The program sorts the percentages from highest to lowest, and sorts the ID's so it ensure that the ID's match the sorted value (see below), generates a random value, and if the value lands between any of the percentages, it selects a set of properties to give to a door. An example of what was happening was this: if I set the probability of a door to be hot, noisy, and safe, to 100%, it would not land on that, instead it would land on a different set of properties.

Before Sort:

Hot Noisy       Safe Door           Percentage of Doors     ID
Y Y              Y                             0.05                       1
Y Y              N                             0.10                       2
Y N              Y                             0.03                       3
Y N              N                             0.21                       4
N Y              Y                             0.06                       5
N Y              N                             0.11                       6
N N              Y                             0.40                       7
N N              N                             0.04                       8

After Sort:

Hot Noisy       Safe Door           Percentage of Doors     ID
Y N              Y                             3                          3
N N              N                             4                          8
Y Y              Y                             5                          1
N Y              Y                             6                          5
Y Y              N                             10                        2
N Y              N                             11                        6
Y N              N                             21                        4
N N              Y                             40                        7


What not to do: Panic, immediately change code, blame yourself.

Changing code, panicking, stressing, etc, are the last thing you want to do because you're wasting your time. In fact, if you really have your heart set on being a bad debugger, make sure in your critical functions, such as setting door probabilities, you have zero (or meaningless) comments, poor naming conventions, and no plan either. When it comes to a bug like this, if you panic, try to change some code, slap some stuff together, or try to hack it, you'll waste a lot of time and not realize what you're doing.

What to do: Recreate the problem, search for patterns, and hypothesize.

When it comes to a problem like this, where probabilities are not setting correctly, it becomes very tricky because it is most likely you have a logical error somewhere. The best thing to do is make sure, if you don't already, have a plan for how your program should flow and what each block of code should do. Describe and justify your code! Try to break it by thinking of how it could not work under certain cases. This can really help wrap your head around the problem.

Next, play with your text file a bit. Try a few different cases and see if it works perfectly in some cases and not so perfect in others. After that, check the door setting function using break points. In my program, I print out the initially loaded text file, the after effects of the door sorting algorithm, and which random value is generated and where it lands/selects. When trying to solve the problem, this is probably a good place to start.

Actually Solving it

This bug gave me a headache because sometimes logical errors just screw you up big time. The first thing I did was want to understand the problem in greater depth. Here is a general run down of what I did to try and understand said problem:

• Recreated the bug several times to see if it broke in multiple cases with or without a relationship (meaning were cases where it broke related to other broken cases or did they appear to break arbitrarily).
• Analyzing the programs output and trying to see if it is landing on the incorrect set function.
• Going back to my original plan, trying to determine cases where my code would break, and seeing if my case would fit.
• Analyzing my load function of the door.

What was important about solving this bug was enumerating and eliminating possibilities. Very quickly I was able to determine that my script loader was not to blame. It was determining the random value correctly, so it's not that. It sorted correctly but I was able to determine through my output and analysis of the breakpoints that the final value it was getting to determine which set function it used was incorrect. If the value used for selecting which door to use was incorrect, then the problem most likely lays within the block of code where it resides.

As a hypothesis I state that the value used to select the appropriate ID is randomizing correctly, but finding the incorrect value. Here is the block of code associated with ID selection.

Essentially, it generated a random value, creates a new door, selects the random number, then tries to place it within the sorted ID's. After viewing this function several times, it works as intended. The problem doesn't appear to be here. What about the sort function? If the problem exists with the ID not selecting the right door, what if I'm not sorting correctly?

Again, after thorough analysis, the sort function appears to be sorting things as intended. So where could this problem exist?

Taking a step back

A good way to debug poorly is not stepping back from your code to clear your mind. If you keep pushing yourself harder and harder to understand a problem that you simply aren't understanding at that moment, you'll doom yourself to stress and anxiety. This was one of those bugs that I needed to step back from and go back to the drawing board about. I thought very carefully about the sorting function and began to ask myself questions such as, why am I even sorting in the first place, what does it get me, why am I using and ID value, and so on.

What I came to realize that when it came to sorting was that I had an inherit flaw in the organization of my code. Essentially, I had a block of this code:

This was responsible for setting the properties of the door. This wasn't the problem, but it was a clue as to what my problem was. Essentially, I was sorting the doors and NOT sorting these if statements to be recognized by my new sort. Which begs the question, why am I even sorting in the first place? Was there an easier way to get the random number generation I needed without having to write a convoluted mess of code just to make sure the if statements lined up?

The idea then became a mindset of doubt, which is healthy. I doubted my very sorting function saying, what if you're wrong? How can I prove you to be wrong? Going back to the drawing board, I knew that the total value of all probabilities should equal 100%. So why not instead not sort the values and use them as is? For example, if I have two values, 0.2, 0.5, and 0.3, why sort them? If I multiply by 100 I get 20, 50, and 30, and they all add to 100.

I tested several scenarios on paper and determined that by adding value 0 to value 1 in the vector, and then the summation of value 0 and 1 to value 2 in the vector, and so on, I actually get a correct scale in which to generate numbers that will, hopefully, lead to correct  random number choosing. Time to comment out my sort function, write the new code, and test that hypothesis.

It worked!

Long story short, the program worked as intended and help up on several tests just to make sure. Bugs fixed. Hooray!

In conclusion

I hope you learned something about what not to do when debugging. When coding, remember, don't panic, don't cry, don't pull your hair out. Relax, think logically, and do your best. That, and, read the documentation, make your own documentation, comment, and have good practices for naming conventions. Thank you for reading! 

How Not to Debug, Part I

In my past blog posts, I discussed tips and tricks on debugging better and different practices for debugging code. However, I never really get into how not to debug. So today, I'll be looking at ways not to debug, so that you may be a better debugger.

Ergonomics & mindset.

When it comes to working at a computer, ergonomics is always critical. Because debugging can be a hair-pulling task, it is best to remain comfortable with adequate lighting levels. The screen shouldn't be to bright, and the space in which you're working should provide adequate lighting. Programs like Flux can really help with ensuring your eyes do not get strained from lighting. A comfy chair that is adjustable, cushioned, and comfortable, along with good desk space will allow the programmer to not feel cramped and help reduce headaches and stress. So when debugging, if you're not following at least some basic ergonomic rules, you could be artificially stressing your body out.

Debugging is not just a task, it's a mindset. If you're not thinking logically about your problem and instead pulling your hair out, screaming internally about why the $#^% that particular part of your program won't work, you could be debugging wrong. Panicking is exactly what you should do if you want to not debug correctly. In fact, here is a list of things not to do from a mental standpoint

• Panic (probably the worst thing you can short of going postal).
• Don't take breaks, don't drink water, don't talk about the problem to anyone, don't do anything to give your mind a break!
• Berate yourself verbally (calling yourself an idiot, for example. We all make dumb mistakes)
• Physically harm yourself (head desk, hair pulling, hitting yourself on the head. Yes, I know someone who smacks their head when debugging and he gets a headache every time). 
• Point fingers at co-workers, peers, etc. This will only make things worse.
• Get angry at the problem

Sometimes a little stress is good as it can motivate without causing any actual harm. However, if you let stress rule you, you won't be a good debugger, just a stressed out programmer. 

Shotgunning and hacking in code

A term I first heard from one of my TA's, shotgunning code means when you just write code into your program and hope it works, without actually looking into the problem. Sometimes, this approach will work, especially for smaller programs, but you won't figure out why and you certainly won't learn anything. The problem with code shotgunning is that you can spend hours just spraying if statements here and there, tweaking little values, adding in new junk, and by the end of it even if you got it working you probably won't understand how you got it to work. 

Code hacking can be just as bad, if not worse, because it turns shotgunning code into a bad habit. Sometimes programmers will encounter a problem and discover a fix but not understand how it fixes it but use it anyways. That isn't learning how to debug, that is having a bag of trick to fix niche issues you don't truly understand. 

Comments, documentation, and code structure

If you want to be a bad debugger, do these:

• Don't comment, or not often enough.
• Have weird variable names.
• Spell words incorrectly.
• No documentation or poorly written documentation
• Don't plan. At all.
• Don't have any flow charts, diagrams, or any indication of how your program is structured. 

If you don't do these, you're not helping yourself or anyone who could be looking at your code. Commenting code is so critical to writing code that will help you fix problems it isn't funny. How many times have you written some code into the wee hours of the night, didn't comment, then came back and looked at it and you had to figure out what it did, just to jolt your memory. If that sounds familiar, then try commenting more with additional clarity. Comments are meant to help someone understand the who, what, where, why, when, and how of code so programmers can understand it easier.

Good advice, but you don't HAVE to follow this exact method

Weird variable names and proper spelling is quite obvious. If your variable names are spelt wrong, programmers will have difficulty finding it. Weird variable names simply means if you're supposed to have a bool that checks if the player goes first during a game of Tic Tac Toe, call that bool isPlayerGoingFirst, not playerMove, or isPayerGng1st, because that will down right confuse people.

I don't think I can stress how important documentation is to code, more so third party libraries or a tool you made for a group project. SFML is an example of good documentation. It has lots of comments and describes how things work in short, brief statements. Nothing bad about longer explanations, but this works in a crisp and concise way. If you are going to code a menu creation tool, or a sophisticated object loader, comment about it and describe how it functions. You'll be happier for those comments if it has a problem.

Next step, plan. Seriously, plan your code. Buy a white board, markers, and plan a flow chart for your code. Don't know how to make one? Google it. Having a logical flow chart of how your program should go from point A to B to C and so on is stupidly important that if you don't do it, then it becomes harder to understand how your program works in the bigger picture of things. 

This isn't hard to do.

Conclusion

It is possible to write an entire book on how not to debug, but this is just a taste of how not to debug. Next time, I'll be discussing about how and how not to test your code by using actual examples, more in depth into how not to code, and some good practices I've learned over the years of coding. 

Star Citizen and Bug Smashers, Part II

Last week I discussed tools that were used by a Cloud Imperium Games Corporation to help debug code for their game. Cloud Imperium's YouTube series, Around the Verse, gives people an inside look into the company, and even a look at how the debug their code in a segment called Bug Smashers, hosted by Mark Abent, a gameplay programmer at Cloud Imperium. This week, I'd like to discuss debugging techniques used from two episodes and juxtapose them with techniques learned in class. I will be focusing more on the techniques and thought processes used to solve the bugs instead of the tools used.

Bug #1: Lens Flare methodology

In Around the Verse, episode 15, the focus of Bug Smashers was on a bug with the flare for engines not appearing. Before hoping in the code, he at first begins replicates the problem both in the game and the engine test scene. Stating the problem, lens flare archetype does not work in the game or engine, and understanding how it happens, he begins to do some fact gathering by understanding it only happens with the ship engines. Double checking the database tool Mark has available, he checks to see if the flare works as an archetype, and sees that it doesn't work. Next he checks it with a normal light entity and sees that it does in fact work.

He enumerates and narrows down the possibilities by looking at the entity initialization, and because Lua deals with the lighting logic, he was able to determine a likely spot where the problem may exist. After narrowing it down to a likely spot, he sets breakpoints to help determine the problem and discovered the properties for the newly created archetype does not set the property tables at all. Writing two lines of code to make sure it initialized correctly, he tested to see if that fixed the problem. The test was a success and the lens flare works in the engine test scene, but it failed to spawn in the game.

Fixing the lens flare in the game test mode was dealt with in a similar manner. Knowing the problem, Mark checked the database to see if there were any anomalies. Seeing right away that a material was not being set for the lens flare, he set the right path and gave it a test in game and found it worked. With the lens flare working both in the game and the engine, the bug was fixed. 

Bug #2: Texture error methodology

Video games cannot display high resolution textures all the time and will instead draw out low resolution textures when higher quality is not needed. In Around the Verse episode 5, a bug was causing the high and low resolution textures of one of their space ship models to constantly flip, causing frame rate drops and very noticeable texture flickering.

Initially, this bug was looked at by one of the graphics engineers who discovered that the level of detail (LOD) calculations were being done twice. The program would calculate the LOD once, then use the same values to calculate them again, causing the switching. The changes incurred by the calculations were causing a floating point error where the affected variables would change so minutely, between 1 and 0.

Trying to replicate the problem, Mark noticed that the bug was only present on static models. So if you have a flying ship, it would flip between high and low textures properly. A static ship statue however, would flip textures. Next, he enumerated the possibilities by looking at all of the related segments of the LOD code and then began to use breakpoints to see where discrepancy happened and eliminate sections where it could not have happened. After finding a segment where the LOD's rapidly change, he checks the call stack to see where the LOD is calculated. Working with the graphics engineer, they were both able to determine the location of the problem and ensure the LOD calculations were using the correct calculations. Once the LOD's were changed to calculate correctly, a test was done in the engine test scene and the game to determine if it was fixed. The test was successful.

Comparison to methods taught in class and key takeaways. 

In both bugs, Mark follows a clear method for uncovering and solving bugs that follows a similar methodology of the Sherlock Holmes method in the class textbook, Debugging By Thinking. Here is a list of methods that I picked up from the Bug Smashers portion of Around the Verse:

• Define the problem clearly.
• Gather facts about the bug.
• State a hypothesis
• Enumerate and eliminate possible bug locations.
• Make a change somewhere.
• Run and test all cases.
• If failed, go back to step 2.
• Remain calm during all of the above.

When debugging problems, it boils down to your tools and technique. It is important to remember that it is not what you think, but how you think. What I find awesome about the Bug Smashers series is the clear methodology used to identify and solve problems. Reminiscent of the Sherlock Holmes method of thinking, the way of thinking presented follows a clear logical path that can used to test any bug. Software defects are not solved by shotgunning code, guessing, or trying to hack in some cheap fix, but by a set methodology that makes it easier and less stressful. 

In class, we are taught about how different debugging tools are used to maximum effect. Bug Smashers also demonstrates the importance and effectiveness of tools when combined with a methodology for identifying and squashing bugs. Tools should be easy to navigate, visually simple, and provide ample information about what you are currently working on without being to invasive into your program. 

I took away three key concepts from Bug Smashers. One, you should have a logical methodology that helps you break down a problem, identify it, and not only solve it but understand why it was broken and why what you did fixed it. Two, your methodology should complement the tool you are using, which for me is the key take away from that portion of Bug Smashers. Three, don't panic. Not once did I see a sing of panic, stress, pull hair, yell, curse, or anything negative, during Bug Smashers. It is best to be in a calm and comfortable state of mind when debugging big problems.

Conclusion

Bug Smashers is a very fun an interesting segment of Cloud Imperium's YouTube series as it gives you an inside look of what it is like to debug for a game in development. There is a lot to learn from the series as it provides the viewer with some real case scenarios of what can go wrong during game development, how bugs can affect the game, and how to go about fixing them. Thank you for reading, I hope you enjoyed it! 

Star Citizen and Bug Smashers, Part 1

After a long bout of writers blog and researching some new debugging techniques, I finally decided what to write my next blog on. Rather than discussing a debugging technique, or a particular tool, today I thought it would be more useful and interesting to dive right into how industry professionals go about debugging their games and how we can learn from it. So this blog will be split into segments, the first will be a tool analysis, the second a technique analysis, of the debugging methods used by Cloud Imperium Games Corporation, the developers of Star Citizen.

Star Citizen and Around the Verse

Star Citizen is an upcoming space trading and combat game developed using the CryEngine. The game is truly massive in scope, featuring first person space combat, first person shooting, trading, building of ships, on top of that a persistent multiplayer universe with a galactic economy, NPC's, and more. The game is huge and debugging is an even bigger task. Every week, the developers give players an inside look into the game with a YouTube series called 'Around the Verse'. An interesting segment of the 40 minute YouTube series is called 'Bug Smashers', where one of the gameplay programmers, Mark Abent, will answer fan questions or debug actual problems in the game. What makes the segment so awesome is the combination of tools, techniques, and experience, that are used to, well, smash bugs.

Bug Smashing in Bug Smashers

What makes Bug Smashers such a fun segment is how well it connects to not only in class lectures, but the class textbook, Debugging By Thinking. In episode 15 of Around the Verse, a bug with the engine flare not occurring during ship engine boost is the focus of this weeks Bug Smashers. Upon a quick test in the games debug mode, we get a visual representation of the problem. It's quite simple, no lens flair from an archetype entity. Note: an archetype entity is based off a regular entity that specifies individual parameter values for that Entity. Very quickly, we get a peak at some of the cool tools necessary to debug problems like this.

The tools

First thing to note was the ability to hop from a test mode to an editor that feels a lot like a Maya scene quickly. Very easily was he able to test flying the ship, to a debug test scene. Next was that we get a look at a menu that displays a database that allows the programmers to easily keep track of 'archtypes'. This data base keeps track of all kinds of data, whether it be light information, entities to use, etc. So if some particular light gets set up in a certain way, the database will hold onto that for easy use to grab later on.

Next, was that this database was used to grab a lens flare entity that should glow orange. It was as simple as a few clicks to grab it and place it in a test scene. What is important to note is that because this is a database, any other programmer focusing on this problem can simply open up the scene and grab that same lens flare entity and begin testing. While that is awesome, there is more to it.

Thirdly, was when he was attempting to solve the bug, the game uses Lua scripting for lights and Visual Studios for compiling code. Every time the code would compile, he would reset the editor, hop back in the level, and go right back to testing.

Exploring the bug

Over the course of the video, Mark discovers a series of problems which he is able to quickly and efficiently narrow down because of the tools at his disposal. After discovering the Archetype was not receiving the proper information, he double checked the code and made sure that the Archetype was getting the proper initialization that it needed, and reran the scene. After recompiling the code, he discovered that the lens flare was working in the editor! Next, after testing it in a debug test scene of the game, he discovered it didn't work. Using the database editor, he was able to identify that the correct material wasn't being set correctly, which was a simple as inputting the proper file path for that material.

Why the tools are so critical

After watching the video, it was abundantly clear why these tools were so useful to the development of the game. The database was easy to use and visually appealing, with ample features. Essentially a list of all kinds of information, programmers would be able to easily create new entities or archetypes that are saved in the database, view critical information, change values on the fly (for example, brightness, color, direction), spawn copies, see material paths, and more.

From a debugging point of view, when running into problems, it is easier, faster, and less stressful, to narrow down and test problems that would otherwise take forever to debug. This can be used to quickly replicate and analysis potential bugs without a steep learning curve of the tools. The ability to quickly change entities and test them right away in a test scene is a very powerful feature. It cuts down a lot of waiting time and gives programmers the power to debug their code on the spot. When waiting time and stress is reduced, everyone wins.

What is so important about these tools is that they are organized, visually appealing, easy to use, and informative. When a tool has these properties, they can help in so many different ways that it will really help the team solve problems much quicker. When tools are lacking in some way, they can be stress inducing, cost the company more time and money, and possibly even prevent some bugs from being exposed, which is never good.

What I take back from this

When developing my own tools for games in the future, a database is something I would love to create. For my Capstone project, I want to create a database to view and edit story instances of the game, so that it is easy to not only set data for them, but to connect them all together. While the test scene might be a bit to much beyond my scope and not fully necessary, a database of all my information in an easy to access and visual way would be exactly what I could use to speed up my production. While it may take a lot of time to make a database editor for the project, the amount of time, and stress, it would save over the long run would surely make up for it.

Next week I'll be discussing the thought process and methodology used to narrow down the bug, as well as its in class application, how it relates to portions of the textbook, and how the tools made all of that possible. Thank you for reading, hope you enjoyed it.

Building a Logger for a Custom Particle System Generator

In my previous Debugging class, my professor posed a rather interesting question: how would you build a logger? What constitutes a good logger? What should a logger display? As with most things in life, it depends. However it still is something to think about. The question really got me thinking, more so because I have been working on my own special effects program that creates and updates particle systems. If I were to create a debugging logger for a program designed for said program, how would I do it and what would it include?

My experience with debugging loggers

During my first and second year of making games at UOIT, the only logger I became familiar with was the Visual Studios error log and console. The error log allowed me to see blatant errors and warnings, while helpful, can only go so far. The console was useful for using cout to display fail/success of loading, if something couldn't be found, and more. The console proved used but was still very limited.

During third year, we used an engine which utilized a logger, but lacked a console. It would print relevant information to a text file and not overwrite information from previous program executions. The logger had some useful functions, as I would print out relevant game information. While useful, I felt it lacking, and I knew there could be more to it to help speed the development process.

How I would build my own

When I would make a logger, it would have two primary components: a printed element and a visual element. The printed element would print information to either a text file, console, or both. This could range from whether or not a third part library initialized, if an asset loaded, if some arbitrary class initialized, or other wise. The possibilities are really limitless.

When creating a custom particle system generator, I need to consider what my program would be doing. For example:

• Number of different particle systems being updated
• If they successfully initialized or not and related information such as:
• Number of particles
• If assets for those particles loaded or not
• The name of each particle system
• If I push a button on the GUI
• If something goes wrong, how long in the simulation was it?
• A generic error function that prints as much who/what/where/why/when info as possible

Printing to a file for debugging

The reason why I'd want to know all of these things is so, if there ever is some kind of problem right away, I can open up my log and have all of the relevant facts to analyze. When a problem occurs, I can save a copy of that text file, repeat the problem, then analyze my logger to try and determine the issue. Here is a mock class I wrote to visualize my logger class.

Obviously this would not be the final product but it might use a lot of the functionality provided. What is important to note is the inclusion of the function error. Error's purpose would be to trigger upon something failing utterly, and subsequently print the who, what, where, why, and when, if possible. It can get the who, where, and what by knowing what part of the program it failed (for example, when loading an asset), when, simply being the time. Knowing why could be as simple as, "it failed to load a file" to something not being properly defined. However, the program may now always know exactly why, so sometimes we may have to take that part with a grain of salt, depending.

Printing to the screen for debugging

The visual element would display live data on the screen or to a console. This data could range from player position, CPU usage, FPS, number of entities within a vicinity of the player, game time, and more. This way, the debugging has the most critical data on screen.

A good example of a visual debugger in a game

Unfortunately a console is limited to pretty much just text. While useful, it overlaps the duty of a text file. I would still use it to print initialize data, but I wouldn't invest to much time in it. When making a visual element for my particle system generator logger, I would want this information displayed live:

• Number of particle systems, and how many are active
• Number of particles. 
• Total 
• How many are being updated all at once?
• How many are being updated by a particular system?
• Inactive particles
• CPU usage
• Simulation time
• Display the information of a singled out particle system.

Here is an updated class:

While it is not much of a change, the display function is going to have a huge amount of responsibilities. It will take in two pointers, particle manager and the active particle system. Using that, it would print relevant information to the foreground, so that it doesn't get blocked by anything on screen. Also, a simple key should be used to turn it on and off. I like to use the '5' key for debugging purposes. 

I would make it so that it prints any encompassing information, such as total particle systems, number of active, etc, to the top left. Below it, would be the singled out particle system data, and to the right, system data such as RAM and FPS.

Example mock-up live display debugging screen

Conclusion

One of the more important things I've learned from my game dev experience is that knowing how to debug is absolutely critical. I also know it can be a painful experience. When making a logger, I would try to make it in such a way that it would make fact analysis and information gathering easier. While what I've displayed is only a mock up, it gives across the right idea of what to think about when making a logger.

Thank you for reading! Hope you learned something.

Debugging by Thinking and Professor Solomon

Previously, I wrote a blog describing my old debugging technique juxtaposed to how book Debugging my Thinking discusses a more Sherlock Holmes way of approach bug finding. However, the textbook describes more then one approach to thinking about debugging then just Sherlock Holmes. The text describes the methods of Professor Solomon's 12 steps to finding lost objects and how you can use the same thought process to locate bugs in code. Today I'll be analyzing those 12 steps, how they can apply to game development programming, and any thoughts I have about them. 

When applying Professor Solomon to debugging, you have to try to make an analogy between lost objects, and software defects. The lost object is an action of lack of action occurring in some unknown location. The visibility of that object is understanding how and where that piece of code causes the symptom. 

The Methods of Professor Solomon

• Don’t look for it
• It’s not lost, you are.
• Remember the three c’s.
• It’s where it’s supposed to be.
• Look for domestic drift.
• You’re looking right at it.
• The camouflage effect
• Think back
• Look once, look well
• The eureka zone
• Tail thyself
• It wasn't you.

Don't look for it

Don't start looking until you have an idea where to look. This is debugging by thinking! Not by hacking code and hoping stuff will work. Don't look at source code once you find a defect and don't look at output once you found or received evidence of the defect. Instead:

• Make a list of criteria that will qualify or disqualify sections of code.
• Make a list of similar defects you have seen in the past.
• Make a list of root causes and try to narrow down possibilities.

Once you have your list, start narrowing down possibilities. Remember, it is about knowing how to look, not where to look.

It's not lost, you are

Bugs are accidental and a behavior that is as yet unexplained. When looking for bugs, it is important you use a logical system for narrowing down your possibilities and finding it. It is important that you use a system for searching. If not, you will waste a lot of time, effort, and sanity, looking for it. Use a system!

Remember the three C's!

To find a bug, you have to be in the right mindset. There have been to many bugs I have chased and chased when frustrated, only to end up wasting my time. The three c's are:

• Comfort
• Calm
• Confident

It is important to be in a healthy state of mind and body, especially when tackling those bugs that has taken up days of a teams time. Drink plenty of water, breathing exercising, listen to calming music, take short breaks, proper lighting, etc. When you are physically and mentally capable of conquering bugs in code, chances are you will find them faster.

It's where it's supposed to be

When a piece of code is causing a bug, that bug isn't moving anywhere. It is exactly where it is supposed to be. But how can we find it? Using deduction, we can find where the bug IS NOT located. Try to narrow down locations where you can know, for certain, that the bug is not located, you can save yourself a lot of searching and stress. Remember, the bug is exactly where it is supposed to be, you can use that logic to narrow down where it isn't, and find it.

Look for domestic drift

Most bugs are normally found in the place you last modified. However, that does not mean you should right away look there. Use deduction to determine the root of your problem. When there is a defect, it will generally have a root point where it defects, then it will cause a trickle down effect where it causes a sequence of problems. Look for those kind of drifts. As a tip, try making a flow chart that illustrates the flow of data in your suspected area to try and narrow down what might be causing the issue. 

You're looking right at it.

During third year, I had a bug where items were not being properly parented. I had skimmed my code a hundred times but didn't see the problem. When trying to locate a bug, remember, you may be staring right at it and not realize that is the bug! Before looking for bugs, try explaining the bug to a friend or co-worker. Write down, logically, what is supposed to happen step by step and eliminate impossibilities. Narrow down your problem to a section of code and look viciously. 

The camouflage effect

I do not believe I have personally experienced this problem, but it is something to beware of. If you are absolutely certain you have narrowed down a problem to a key area, but cannot determine what may be causing the problem, consider that there are other factors that can place a problem out of view. For example:

• Preprocessor macros
• Procedure Calls
• Exception handling
• Complex language constructs (C++)

Think back

This section only applies to code you created. Try to think about what you did in terms of changes, and how that would alter the flow of logic you were intending to create. What is best for this section is to create versions of your code frequently so you always have a backup to look at. Also, commenting your code frequently, and providing large blocks of comments that describe changes, helps. It can give you a good idea of what you intended to do which can subsequently help you narrow down a list of problems you should create. 

Look once, look well

I wish I followed this one better. I mentioned earlier I had a problem with parenting in a game I was making in third year. I looked in so many different cpp files that it was giving me headaches. What was worse, was that I looked at the same spot several times, even when I thought it wasn't the problem area. When it comes to debugging, look once, look well. 

At each potential problem area, use all relevant tools to help find the bug. As mentioned before, also keep different versions so you can always go back and look. However you have to be wary when taking this advice. Make sure that, when you do look, you either confirm or eliminate the possibility of the problem being in that location. If you simply do not find the bug, check your other potential problem areas. If you know the problem is in that area, look again and try to understand the logic of what the program is supposed to do.

The Eureka Zone

More often then not, bugs develop where they should, others occur in the immediate vicinity. The textbook describes two types of displacement, physical and temporally. Temporally displaced problems are described as:

• Recently modified variables
• Recently executed statements
• Procedures closest to the call stack

Physically displaced problems often have to do with pointers. This is what the textbook suggests:

• Suspect variables in the same heterogeneous storage construct as the variable in which the problem became visible
• Suspect references that are on the call stack at the same time
• Suspect references to storage allocated on the heap at about the same time as the variable in question. 

Tail Thyself

Certainly the fanciest sounding tip and perhaps one of the most important. The concept is simple, tail exactly what you did and how you got to the point you are at. Keep versions, code comments, flow charts for logic, even if you have a written description of what you wanted to achieve, anything that can make you follow in your footsteps to help you narrow the problem down. Using revision software is critical for this step.

It wasn't you

After your herculean effort to try and figure out your problem, it may be reasonable to assume that you're not the cause of the defect. This one only applies if you're working with other people, or some other software development kit. With this problem, ask questions and do some looking around to see who or how the problem came about. If it is someone else in a group, then simply ask them. But make sure to have at least tried to narrow down the problem and create a bug report that is useful to that person. If you're using a SDK, then create a test case to demonstrate the problem and ask a thorough question. No matter how dumb your problem may be, never be afraid to ask.

Conclusion

When debugging, it is important to look at bugs as lost objects that you have to find. Using simple logic and a systematic way of finding bugs, you can reduce your stress, increase your productivity, and simply have more fun coding. I would say the most important take away from Professor Solomon's tips are to have a system for searching and remaining calm, comfortable, and confident. Thank you for reading, I hope you enjoyed my latest post!