Another reason to keep backups... debugging

A bit of an unusual CC progress update today. No new tracks down, or scenery, or fixing that gap in the mainline I spoke about in the last update. That is not to say there hasn't been any progress. I have been adjusting the terrain, which I have described in Part 1 and Part 2 and soon to be Part 3, which will be the largest article on the subject to date. Check back later this week for that.

What I really want to talk about is debugging. Its a huge subject which can be handle many different ways depending on what the problem is, your style of "divide and concur", or if you just good at deduction. Even though I am a Software Engineer by trade and work closely with QA to write Test-driven code, of which I am not going to bore you with, I tend to take a "divide and concur" approach to debugging. In this case I knew I had problem with my route which did not exist in the near past.

The issue I was seeing is that my asset count per tile went up by about 400 objects. This occurred on empty tiles well off the map as well. For those that are new to route development (such as myself) it is important to keep your asset count under 1000 or suffer from slower loading times and drops in frame rate as you enter the tile. Where did the number 1000 come from? Experience from other route makers. It is spoken about frequently on the forums.

Since I knew this was not a problem in the past (I have screen shots of when the count was normal) I started loading up my backed up route files to see when it started to happen and would prove if it was indeed something with my route or something with the game system. Fortunately this lead me to my April 26th backup. This confirmed the problem was my route, and now I had a base line to compare against. I made a (manual) clone of my current route by altering the folder name and route properties, and then adding in the April 26th route using the same technique.

My first test was against the latest changes I made to the route with the terrain. Rolling those back changed nothing, so I knew the problem was due to any of my new additions to my provider/product folder. My second test was to see the difference between the routes at the file level, of which there were several. I swapped out some files that seemed overly large or small (thinking there was some corruption in the BIN). Still nothing. The third test was to launch using the LogMate tracing application to see if there was a difference between the two routes when they loaded. This did uncover something. The bad route was trying to load from a blueprint (WCML North) which that was not formed correctly. I don't know where I got those files, though they may be related to the "Port Road" route which was by the same author. I tried to deselect the objects from my route, and then I forcefully removed any files made by from Keith Ross in my assets folder. This ended up changing the load time for the CC, but it did not fix the asset count problem.

How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?

~ Sherlock Holmes

After banging on this for a couple days, I started chatting with MadMike. He is a good sounding board when I get wrapped around a problem, and this was getting to feel like "work" instead of "play" and let’s face it, I want to play after a long day of work. After listening to me walk through all my tests and the minutia of my file level exploring, he suggested that I check the Scenarios folder. Though he did not had the same problem I had with the inflated asset count, he had issues in the past with a scenario which lead to "unusual" behavior (eventually leading to a crash). It was the one folder I had not taken apart because I didn't think it had an effect on the assets of the route. What could I lose by swapping out the bad version with the good? Nothing. I cloned the route specifically so I could take a hammer to it. As it turns out the scenario (I only have the one free roam to test with) was the problem. I have three options now. I can either diagnose the dozen files in that folder and hope to find the root cause, or I can swap it out with my backup, or I can blow it away and make a new free roam scenario.

At this point I will spend just a few more hours digging into this. If I find the cause I will share my findings so others can learn from such an unusual problem, however; there is something else to be learned about this experience which is without backups I would not have been able to do a lot of this diagnosis and it would have taken much longer to come to suspect the Scenarios folder because I would not have been able to rule out the other files and folders through testing.

Update: Mystery Solved!
There were less files in the Scenarios folder than I thought. Most of them were backup files from RWTools. I narrowed the problem down to my Scenario.bin which contained all the rolling stock I added to the route. I removed the scenario so RW would create a clean one, and noticed when I started to add rolling stock that the asset count increased for all titles. Even more interesting is that a model may take be made up of more than one asset. I confirmed this by loading up another route (NSAND) and in the yard the asset count started well over 1k and off the map it was just over 800. I don't feel so bad anymore, and I can work around this by having RW create the clean default scenario and use it only for scenery, and then add back in my older scenario and use it for fun.