Frustrations on programming languages

So, I'm honestly a bit demotivated to work on Magecrawl. Part of it is that I'm in the middle of porting it to Silverlight as a way to get a tiles version with little work that will work on Windows/Mac/Linux. Then this week, I come across these articles.

While it isn't like they are canceling Silveight, their "change in focus" and the fact that Mono is currently two major versions behind has me a bit worried. I don't want to sink another 20+ hours fixing things up, and then in a year or two have to move to another UI layer. I also don't want to loose being able to work on it on my Mac, and right now I'm in that boat.

More fundamentally, I'm frustrated that there doesn't seem to be a "good" way to do cross platform UIs for my game. I'm going to rattle though the programming languages I know or are good options. None of them right now seem to be a good fit.

C# - Silverlight: Only Mac/Windows right now. Linux is an unknown period of time away. A bit worried about its future, but probably best option right now.

C# - libtcod: Works everywhere, but no tiles UI.

C# - winforms - Yeah, it might technically work on Mac/Linux, but there are licensing concerns that make ppl not want to install it.

C# - GTK# - Doesn't look native on ANY platform except Gnome.

C/C++ - Any number of frameworks here, QT being the most obvious framework or OpenGL. Ignoring the rewrite aspect, I'd rather not chase memory errors in an unmanaged language when I'm not being paid for it.

python - pygame would be the most logical choice. I don't think I want to write in a dynamically typed language.

objective C - See C/C++ for the unmanaged language type. It also wouldn't work well on Windows/Linux, and I have no clue what GUI framework to use. 

Java - Seriously? Swing looks terrible anywhere, and with Sun Oracle trying its best to kill developer uptake, that isn't an option.

In writing this article, I've come across opentk, which appears to be a open gl binding for C#. While I'd have to learn yet another UI framework, at least I know it'd work everywhere...

It's amazing I wrote some of this old code...and yet I don't regret it.

Another of the mini-goals of this iteration was to provide a "Welcome/Title" screen to magecrawl, allowing the player a place to enter a character name and choose between save files. As I might have mentioned before, this "attempt" at Magecrawl is not my first. I've written parts of my game before in C/C++/python before a settling on C#. My wonderful wife has heard more than one time about me throwing everything away and starting again (or wanting to). I had the title/welcome screen from a previous attempt laying around in an old svn account, so I figured I'd salvage that as a good place to start.

My goodness, if I didn't know for sure I was the author, I'd want to disown this code. Ignoring the horrible style of capitalization and spacing I must have once liked, I've found multiple WTF moments. See this. One involved a corner cases with the color scheme that sometimes overflows into an exception. One was a half a dozen lines of code trying to reinvent Path.GetFileNameWithoutExtension(), and doing it poorly. The other memorable one was me declaring an enum to record what menu item we were pointing to, then ignoring it in a half a dozen places and comparing directly against (0,1,2). I hadn't given the enum a numerical set of values, so it just "happened" to work.

There is a proverb in Go that says, "Lose your first 50 Games as quickly as possible". The idea is that in the beginning, you don't know what you are doing, so don't overthink your moves. After 50 games, you can begin to see what looks like a good move and why people play as they do.

I'm starting to think the same idea might be valid for programming a roguelike as well. Experience has taught me, and while I wouldn't suggest throwing all your work away over and over again (like I did), just getting out there and coding will make you a better coder. While I haven't entered a 7 Day Roguelike context myself, it sounds like something I'd suggest and should do myself in the future.

The evils of duplication and DRY...

For those who've taken formal software engineering education or have programmed as a career, this post might something you've already heard of. However, I found myself coming back to this principle again while working on some code, and I figured it deserved a post nevertheless.

DRY: "Don't repeat yourself". Originally formulated in Pragmatic Programmer (a great book I read as an intern), this guideline for software development is simple and to the point. The idea is that you should never have two places that contain duplicate representations of any given fact.

The most obvious implication of this is that you shouldn't copy code. If two places need a given chunk of functionality, abstract it to a base class, static method, helper class, etc. At work, I've heard this copying known as the "editor inheritance" pattern, the "editor" being your text editor.

Even if you don't copy paste your way into this situation, it is easy to run into in other ways. In magecrawl, the two ways to invoke a spell-like affect is via the "Cast Spell" screen and selecting an inventory item and selecting the use verb. I found that keyboard handler for both these screens were 99% identical. In the beginning, spells and spell like affects had no targeting possibilities, they always targeted the player. As I added targeting, I added it to both places not realizing the duplication. This became a problem when I wanted to add a new targeting type later on and nearly missed one of the two copies.

I find that when I find these areas, the best thing to do is address it either "right now" if your in development, or as soon as the next release occurs if your stabilizing for a release. I shelve whatever changes I was working on, and figure out how to remove the duplication. In my case above, I created a base class that contained the common code and had both classes inherit it. I then tested it thoroughly and submitted it before working on new features. Your refactoring hat and your new feature hat shouldn't be on at the same time, you look silly when you try. :)

The more advanced implications of DRY involve not repeating yourself in more than just near identical code. The book mentioned above handles some of those cases, and maybe I'll come back to them in some future post (when it isn't nearly midnight as well).

Non-English operating systems, InvariantCulture, and You

As previously mentioned, Magecrawl's 2nd tech demo was released yesterday. I received multiple reports of crashes during map generation. After some detective work, it became clear that all of the reporters were using non-English versions of Windows (Polish and French). This issue can be boiled down to this simple string:

"10.0"

In some languages, the period there is replaced with a comma. All of the .NET languages by default, defaults to using the system language. This creates problems when text written using one separator is read by a program using another. The text is read fine, but when you try to parse it to a number, an exception is thrown.

The solution easy (arguably hacky) solution that somebody showed me is to tell .NET to "stop doing that".

            CultureInfo previousCulture = Thread.CurrentThread.CurrentCulture;
            Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
            //Your code here that depends on reading/writing text between languages.
            Thread.CurrentThread.CurrentCulture = previousCulture;

What we're doing here is saving off the current thread's "culture" and setting it to InvariantCulture. Then we can do our operation, and reset it back when we're done. In my case, I wrap the functions save/load functions and those that read spell/item/monster information.

If you've been bitten by this issue in Magecrawl, grab the new release here. If you use an English OS, there isn't a behavior change.

Easy version control - Why you should submit early and often

So the changelist that brought writing this post to mind was this one. I was working last weekend on Magecrawl, and decided that writing a "lightning bolt" like effect shouldn't be that hard. It just travels in a straight line until it hits a wall (or travels a maximum distance), hitting every creature in its way. In developing it, I violated a rule I try to follow in development both at home and work.

• Each change should only do one things (Add feature, fix bug, fix performance issues)
• Corollary: Submit each change independently as early as possible

It started off with the basic infrastructure for a new effect type, and then it ballooned into multiple different attempts of writing new subroutines to calculate the path of projectiles. After that, I didn't like the way it looked when drawn, so I refactored how we drew all ranged attacks on the screen. I also tacked on some unrelated preference settings, and some bug fixes.

The reason this was a problem was that things took twice as long as they would have if I was disciplined. Keeping changes small makes testing them before submission easier. It makes reviewing before submission easier (I review my changes for submission to keep myself from submitting lazy hacks). After each part, the lightning bolts still weren't right, and I got discouraged from hacking on them further.

Submitting early and often forces you to keep biting off small bits of the problem, fixing or implementing just that part, testing just that small part, and getting it submitted so you can work on another. I find that it works well for me.

libtcod-net 1.5.0b2 release

(Your regularly scheduled roguelike posts will continue after this brief announcement.)

Magecrawl uses libtcod-net heavily both for graphics display and in the game engine (pathfinding, LOS, etc). libtcod-net is just a thin c# wrapper layer around libtcod that I wrote before starting Magecrawl.. I think for any new roguelike, written in either in a managed or unmanaged language, libtcod is a wonderful library to use. jice, the writer and maintainer, is very responsive with suggested features and bug fixes.

A new version of the library has come out in beta, 1.5.0b2. It adds random name generation, sub-cell image blitting, and a few other things I can't think of right now. I'm not currently in need of any of the new features, but it seemed like a good time to update the wrapper and move Magecrawl to use it.

If your in the market for a library to sit your new or current roguelike under take a look at libtcod and/or libtcod-net.

Not forcing a square peg into a round hole: Stairs

So this story begins with a somewhat unexpectedly, I was working on map generator optimization (here) and rarely I'd hit an exception I wasn't expecting. The exception said I tried to generate a given level 10 times and failed. Digging into it further, the stitch map generator was having issues placing the required number of "chunks", coming up short the number I figure made decent map sizes.

It only did this when I turned off the cave map generator, and generated more than 50 levels in a row. In the end, it was a very interesting bug, so I hacked together a few "paint" pictures to explain what happened.

The stitch map generator has a "canvas map", sized 250x250, that "chunks" are placed upon. Stairways were initially very simple, a mapobject that only kept track of if it was "up" or "down". The map generator placed an "up" on one level at the same x,y position as the "down" on the level above.




Here, we generator a map in the box, and place the stairs down at the dot.







Here, on the next level, we try to generate a map centered on that "down" stairs. We hit the edges of the "canvas", and have to generate a smaller map. We again randomly place the stair going down in the lower right.

Here, we try again to generate a map centered on the "down" stairs. It's at this point that we can't generate a valid map and give up.




While in this artificial example it took 3 iterations, in my bug it took 20+. Also, when the "cave" map generator was used 50% of the time for a level, we'd have a large number of spots in the "center" for the downstairs, which could reset the progression.

The correct solution for my issue was to decouple the stairways from requiring the same x,y position between two levels. The naive solution was to add a few properties to the Stairs to keep track of both floors positions. This was very problematic. MapObjects, which stairs is a subclass along with doors, cosmetic items, and treasure chests, are expected to be located only on one map, in one position. I could add logic to figure out what level we're on when we ask for the position, and return that. However, what about levels being generated, and didn't have a known floor yet?

After about a half an hour of hammering a square peg into a round hole, I changed my approach. MapObjects were expected in one position, so keep them that way. I gave each stairway a unique id (guid), and created a class that stores where that stairway leads. I had to serialize this information for save and load, but beyond that it was straightforward.

Now, stair's positions are decoupled between levels. I ran a few level generations of 1000 levels, and haven't run into the original bug case. This also simplified both my map generators. On average, it also speed up the cave map generator, since we don't have to restart when we don't have clear floor where we require a stairway.

Evaluation Loop - Why build and setup times matter

Lately I've been thinking about "turn around time" and programming. I'd define this is the time it takes one a change is made to determine if it is correct. There are two parts to correctness:

• The syntax is correct and the compiler will accept it.
• The code "does what you want"

The most important resource you have as a developer is your attention. The shorter this cycle is, the more likely you are to not get distracted and get more work done. This is why in magecrawl, I've done what i can to make the turn around time as close to zero as possible.

• Visual Studio will underline bad code as you type, so you don't even have to hit compile to know you forgot a ';'.
• C# has a ridiculously short compile and build time. I can build my entire game in less than 3 seconds. 
  
• The build step compiles and copies all the associated DLLs into a "dist" directory which has all the associated data files checked in. (No manual setup on new branches/machines).
  
• Debug builds turn off the introduction screens, "are you sure" timers, save on closing window, and anything else that slows down testing.

There is one things I'm currently not doing which might reduce "turn around time" even more.

• Unit and integration test the game engine and utilities module. This way, I could check common use cases with the hit of a button.

My excuse for this is lack of developer bandwidth. Ironically enough, something that could reduce my testing burden is too much of burden right now. As a developer, if other people aren't using my code, I tend to loose interest after awhile. That's why I released a tech demo of magecrawl, and want to get another tech demo finished soon, to get people to use it and keep my interest.

If you are programming at all, it might be worth investing time into making your build more automatic and faster. It's not the time savings as much as the attention savings that are paid back as dividends.

Optimization II - When the difference between struct and class saves you

So, I'm still on an optimization kick. Map generation is still taking way too long, maps take too much memory. I figured I'd share this story, since it wasn't what I expected.

So, I want the ability to have large levels, even if I don't use them. Say, 250x250, and maybe 50 levels total. My original data structure looked like:

class Map
{
     private MapTile[,] m_map;
     ..more..
}

class MapTile
{
   enum terrianType;
   bool visited;
   int scratch; //Use in map generation algorithsms.
}


There are two major issues with this, naive set of data structures. The first is that we have an array of classes. In c#, this means we really have an array of references. It looks something like this poor graphic.








This is a bit wasteful in my case, as it added 4 bytes (on 32-bit machines) for every map tile. 250 (tiles) x 250 (tiles) x 50 (levels) x 4 bytes = Around 12.5 megs. So switching to structs (which are stored inline) makes sense.

The second issue was the data types for the MapTiles. Ignoring padding, each maptile is (4 bytes (Enum) + bool (1 byte) + int (4 bytes)) 9 bytes large.

In c#, you can set the underlying type for enums. Since there will never be more that 255 terrain types, I switched that to a byte. The scratch value also never needed more than 255 types, so to byte that went as well. My final size is 3 bytes.

For each of these changes I measured speed and memory usages before and after and compared. This way, I could make sure things like structure padding and such wouldn't ruin my day and make things take the same amount of space anyway. I won't share the numbers, out of embarrassment of how much memory was being uses, but let's just say I use a much more reasonable amount now.

Optimization - Or when your application eats up 150+ megs of memory generating maps...

So, yesterday I finished stairways and generation of multiple level maps. It's somewhat boring, but it works. I turned the map generator up to generator 5 levels and built a release build, to get an idea of performance. There was a second pause before the game started, a bit more than I'd like to see. I turned it up to 50 maps, and ran into a 30 second pause that the task manager claimed ate up 150 megs of ram.

Now before you get out pitchforks and tell me that all the managed code stuff I talked about in the previous article is the cause of it, I want to make two quick points.

• Much of that memory would be reclaimed the minute another program asked for it, since it was a lot of garbage the garbage collected was waiting for next collection on.
• The mistakes I made would have been just as slow in C++, RAII would have reclaimed the memory right away (good), but all the allocations I made would have been just as slow (bad).

I won't bore you with the normal two or three quotes about premature optimization and difficulty in predicting where to optimize, you have already heard them before. What I will say is that every time I run into performance, every single time the cause is not where I'd guess.

A month ago when I was just starting the map generator, I was seeing poor performance moving around. I figured it had to be in my game engine somewhere, maybe in the other actors movement. It turned out to be a part of the GUI that was walking every single tile in my map to draw, even if off screen. Since it was in my GUI loop, just the act of walking all the tiles didn't scale to 200x200 sized maps.

To get back at the story at hand, I figured the way I was generating maps was too slow, that I'd need to rip it out and start over. I made the decision to measure my performance and see what was slow. I'd like to tangent to point out the CLR Profiler and EQATEC Profiler, which are free and awesome. So, it turned out that the subroutine that placed chests and monsters was taking 55% of the entire generation time, and using a crazy amount of memory.

The way the chest and monster placement works for my cave generator is simple: from the generated map, get a clear point that isn't too close to the stairs and use it. Repeat for number of chests and monsters. The problem was in the get a clear point. My naive algorithms walked the entire map and created a list of free points. Then it randomized it, and started grabbing from the front of the list until it found one wasn't too close. I figured it'd be more effective than randomly generating points and checking them in maps with a small percentage of empty tiles. You might see the problem here already, I'm walked all the tiles to painstakingly generate a list, randomize it, and then throw it away when done. When you do this 20+ times per level, then 50 levels in a row, it adds up.

Once I cached the list, the generation time dropped in half, and so did memory. I found another similar issue in the core map generator, see here for details. I'm tracking one more down in the stitch map generator right now.

The "lesson" from this story, if there is one, is that I would have never guessed that GetClearPoint was the culprit. A lot of people claim they want fast code, and might try to optimize where they think it's going to be slow, and do nothing but add bugs and obscure coding constructs. Write the simplest code first (with reason, no bubble sorts), then measure and see what's slowing you down. Once you know that, you can whip out the black magic optimizations there.

Why your next roguelike should be in a managed language

So I strongly believe that for new projects, the language of choice is important. It determines what platforms you can target. It can determine your license. It determines the type of code you write. It determines who would be interested in working with you. It determines how quickly your code can run, and how quickly you can write code and find bugs. It is one of the more important decisions you can make, since changing it requires a huge amount of work.

Now I'm going to divide languages into two groups, with a grouping I feel is pretty safe to make:

• On one hand you have unmanaged languages. Those are ones that are generally considered "lower level". 
• They almost always have pointers. 
• They compile down to assembly (or are written in it). 
• They are generally extremely fast when run, with little overhead when carefully programmed in. 
• They are also very unsafe. You can scribble over memory almost anywhere. You can corrupt your stack and crash on returning from a function. You can many times walk off the end of arrays and lists and crash. I generally program in these at work, and some bugs can take days to sort out. 
• Good examples include: C, C++, and assembly
  
• On the other hand you have managed languages. The are the ones that are generally considered "high level"
• They never have pointers, unless dealing with unmanaged code
• They generally compile to a virtual machine or are interpreted
• They are slower they the equivalent unmanaged code. How much depends on the language and what your doing
• They are generally "safe". You get exceptions for doing incorrect things like walking off arrays and such. If you don't catch that exception, your program will still die, but you'll get a nice stack trace.
• There are many example, the most common include Java, Python, Ruby, C#

When making your decision, you have to make tradeoffs in determining what language to use:

• What languages you know - People generally lean towards languages they know for their "fun" projects. Others enjoy the chance to learn something new
• Speed vs. Portability vs. Speed of Development 
• All things being equal, C is the fastest and most portable. However, it's also the easiest to shoot yourself in the foot with. It also doesn't have a standard Graphics and Input Output library. 
• Other languages come with portable graphics libraries, but may be too slow. Also note, that most roguelikes are turn based, so speed is in general less of a concern
• Syntax and language - Sometimes the language itself is not suited for what you want to do with it or is hard to read and write.
  
• Library - Sometime there is a library you really want to use for your roguelike, but it can only be used in a given language
• Note, with many languages, it is possible and sometimes easy to call other languages. C# gives P/Invokes (see libtcod-net), and Java the JNI for example.
  
• Ideological - Some people choose languages to write in based upon a philosophy or outlook.
• Some people reject Java due to it being "too slow", with too slow based upon Java from the 90's on machines twice as slow as today
• Some people reject C# due to distrust in Microsoft and fear of lawsuits, even though there is a open source implementation of it and the lawsuits could only matter if your using the interop libraries.

I wouldn't be honest in saying I don't have a preference. C# is in my opinion the nicest language to work with. It runs everywhere with mono, it is type safe and fast enough for what I'm working on, and the syntax is close enough to c++ that I picked it up easily. The bindings to libtcod make things nice as well.

However, Java is a good language as well, with many successful roguelikes written in it. Python is another language I enjoy to write in, and has multiple good toolkits to write roguelike in (libtcod, pygame, wcurses).

What I'm not saying here is that unmanaged languages are bad. C is the mothertongue of roguelikes. C++ is very popular as well. I use both at work daily. I'm just saying is that for turn based, and low graphic real time games, the benefits may not out weight the costs.

On Tools

In programming magecrawl, I've grown to have a list of preferred tools, and since i have a few moments before bed, I figured I'd list.

Mercurial - Distributed version control is awesome, even when it's just you working on a project. Mercurial is great because unlike git, it doesn't run like a dog on windows. The windows integration is nice as well. I wish the visual studio support was a bit farther along, but overall a great version control system. If you're coding, and not using version control, you should be.

Visual Studio - By far the nicest environment to edit c#. Intellisence always works, unlike in c++ visual studio. Monodevelop is a good standby, and the best on non-windows platforms.

Vim - For all the text editing that isn't code. If you're a vim user, learn macros. They will become your new best friend. For those who don't have the patience to learn an editor with just a high learning curve, I heard notepad++ is a good outside Visual Studio editor.

LINQ is awesome

So, after some prodding at work, I learned enough LINQ to be dangerous. Boy, it makes some code shorter and easier to read.

From:

public bool Operate(Character characterOperating, Point pointToOperateAt)
{
bool didAnything = false;

foreach (MapObject obj in m_map.MapObjects)
{
OperableMapObject operateObj = obj as OperableMapObject;
if (operateObj != null && operateObj.Position == pointToOperateAt)
{
operateObj.Operate();
m_timingEngine.ActorDidAction(characterOperating);
didAnything = true;
}
}

return didAnything;
}

To:

public bool Operate(Character characterOperating, Point pointToOperateAt)
{
    OperableMapObject operateObj = m_map.MapObjects.OfType<OperableMapObject>().
SingleOrDefault(x => x.Position == pointToOperateAt);
    if (operateObj != null)
    {
        operateObj.Operate();
        m_timingEngine.ActorDidAction(characterOperating);
        return true;
    }
    return false;
}

It doesn't seem like a lot, but multiple it a few dozen times and you get some serious space savings. It also helped me realize that the previous code was allowing for multiple map objects in a given square, which violates one of my map invariants.

Plugging Away...

The thing about personal projects is that you generally work on them on your off-time, after 8+ hours of coding and such at work. Sometimes, even if you really enjoy the topic your working on, your too mentally exhausted to come home and code. Sometimes, cooking dinner, a walk with the wife, or any other normal activity comes along and uses the time you planned on coding today. Sometime, when you stare at the list of things your project needs, it can be overwhelming.

The thing about magecrawl is that I've restarted it one too many times. I think I'm on number 8 if you get technical. I'll get so far, get to a hard part, look at my code, and want to start over. Now some of those restarts taught me important lessons: why using a managed language makes things easier, why you should separate the game engine logic from the gui, etc. However, at this point, restarting would be a waste. I don't have any of those project breaking issues.

So the strategy I have for not getting burned out and wanting to quit? Plug away a little bit each day. Yesterday I fixed some issues with the map generator, today I'll write a help screen, maybe next week I'll work on getting more than one type of monster. It doesn't matter if I work 20 minutes, and then go play torchlight, at least I accomplished something.

Now that I've written this out, I feel like it's a poor clone of Fire And Motion from Joel.

Hello World

In the beginning, the programmer wrote "Hello World". And he saw that it was good. Then then the programmer wanted to tell others about it, so he found a humorous title and started a blog. And thus, IfErrorThrowNewBrick was born.

It's here that I hope to write on my experiences on programming in general, and specifically on my work on my pet roguelike game, magecrawl. I've been programming in c# for about about a year a a half at least, mostly at home but now at work as well. It's now my favorite language to write in, with the exception of using python to hack out rough tools.

Well that's enough of an introduction.