Iteration 9 and 10 - The new plan

So, I'm changing my iteration plans around the release of a video game. Let's just put it out there. Final Fantasy 14 is being released September 30th (early access on 22nd), and I am excited enough to pre-order it. Due to that, and the expected week or two post-release that I won't involve much coding, here's the new plan.

Iteration 9 - Finishes asap. As soon as I get the game balance reasonable, and a spread of items/weapons for the 5 levels of the current dungeon, kick it out of the door. I'm planning on calling it tech demo 3.1 or something similar, just for those readers of the blog who want to see the new stuff.

Iteration 10 - Get as much done in August and September. Things like diminishing returns, school specific debufs, sound drawing monsters, etc. Also get enough of the small features done that the next "iteration" will be more of a "slice". Something where I create more interesting gameplay content and less concern on the game engine.

Generic can be the enemy of progress

As it was said on coding horror, "a blog without comments is not a blog". Earlier today Jotaf posted a comment on my last post that got me thinking. The post was on my work making changes to the skill tree easier, here's the meat of it:

"Nice, I agree completely, although you must always ask yourself the obvious question: "is it worth it?". Since you're gonna change the skill tree many times during balancing the answer is yes. This is almost always true for data-driven gameplay features (ie, stats, skills...). It's almost always not true ("it's not worth it!") for things that are more or less set in stone, like game mechanics, combat, AI, and I've seen many developers getting themselves into an endless loop of refactoring, because you can obviously always make future changes easier by changing your design to be more generic/plugin-driven/offloading more data or logic into external files."

I think this is true and in some cases can prevent roguelikes from ever getting completed. Since none of us get paid to work on our pet projects, the temptation is to never set deadlines or abide by them. The desire to make the code "perfect" can make us spent all of our time making the code more generic to the point we're creating a game engine more that game itself. While this can sometimes be successful (ToME being the most obvious example), for 95% of us it doesn't buy us much.

If I spend 10 hours refactoring the AI to move it to external files, and then spend only 5 hours tweaking it over the next year, the change was obviously a loss. Even if I save some time, that needs to be balanced by the fact that the code is more complicated. As opposed to a function you can debug, a problem can be in the file parser, the external file itself, the code implementing the behavior expressed in the file, etc.

I have a few suggestions for other developers based on my experiences with magecrawl that so far have mitigated this issue for me (beyond my general pragmatic attitude).

• Set deadlines, preferably short ones - When you want to finish your iteration in three months, spending a month and a half on something like swig'ing libtcod or converting libtcod's build system to cmake, makes you at least think about the trade off you are making.
• Use bug tracking software - I love fogbugz myself, but the one built into google code or even bugzilla is better than nothing. When you find something you'd like to refactor or make more generic that would take longer than 15 minutes, go add an entry for this or a later iteration and get back to what you were originally working on.
• Things that change often or by users should live in external configuration files, things that don't shouldn't - How often are you planning on changing your physics system? Even if you think this'd more more often than once, leave it hard coded for now. After the third time you mess with it, then go make it generic. By the time that third time happens, you'll probably have a better view on what to pull into data files anyway.
• You can ignore any of these rules if you want to - beyond the fact that I'm just a random blogger on the internet, you are writing your game. While I think getting a working game done early is the best way to do things, realize you are doing it for enjoyment. If spending a year writing a engine that can make toast, run a neural net AI, or show pong on the screen, all with just changing configuration files sounds fun, go get coding.

One final node, I love comments on my blog. They are the breakfast of champions. Please feel free to leave them, they make things more interesting around here.

When making a type of change in your game is tedious, first fix things so that it isn't anymore

So part of my balancing work involved buffing the player's abilities when it makes sense. One way to do this is to add new skills on the skill tree. However, the system I had in place for read/displaying the skill tree made me hesitant to do so, as even a simple change was a lot of work.

The previous system involved a text file with a grid of ascii characters laying out the exact location for each node, along with a list of positions below to match each one up with the associated skill. If I wanted to add more nodes, I'd have to manually place it, then calculate the offset and write it below. If I needed to make the grid bigger, I'd have to recalculate the offset of every node for that tree. It pretty much never should have gotten submitted in that state, but it was "good enough" for the simple trees I had implemented. The horror can be found here for those who are morbidly curious.

I noticed that all my nodes were in a grid, so I converted each text file into an xml file describing the skill tree. One example can be found here. I then wrote the xml reader and which would generate from the stored information the grid of characters to display on the screen. Overall it took about three hours to write, test, and submit.

In less time, I probably could have made my balancing changes and "got out" without making things better, but I would have been more likely to not add new skills in the future. Now adding a new skill is as easy as copy pasting an xml node and changing a few lines of text.

"Backgrounds" - The compromise of class selection and skill tree

One of the unwritten design guides I've followed in magecrawl has been to not ask the player for choices they can't answer before the game starts. The first time I launch a game and am ambushed by something like this (care of Crawl Stone Soup):

I get overwhelmed with choices. What the difference between fighers and gladiators? What is the difference fire/ice/air/earth elementalists? These choices are ones that are difficult to make before playing (multiple) times.

The reason this came up during my initial balancing effort is that I realized balancing some options were difficult. For example, how much should the heavy armor skill be? I didn't want to start the player with equipment they couldn't use, so by default they'd have to scavenge for at least part of a set before taking the option would make sense. It'd need to be powerful enough that it'd be worth going to all that trouble, but not so broken as that'd be the only logical choice.

Letting the player choose to use armor/heavy armor before the game makes some sense, as I could start them with a set of said armor at the beginning of the game. I've decided to try to take some of the nice features of crawl's system and pair them down to something more reasonable:

Opinions? These starting backgrounds would provide the initial equipment and preselect some skills in the skill tree. However, there's nothing to prevent a Templar from selecting skills from a tree and become a ranged attacker, beyond the penalty from using heavy armor.

Current plans on dealing with game balance, with the first step being spelling it correctly...

(Potential bloggers out there, don't write new entries when you are tired or else you might blast everyone's RSS reader with blatant misspellings like my last post)

So as I mentioned, balancing roguelikes is hard since doing so involves balancing multiple variables (with a significant amount of randomness thrown in). In an effort to motivate myself into finishing this work, I'm writing up a quick post on my decisions and strategy.

The first variable I focused on was damage output. I decided on a "base" damage an average attack should do, 10d3. For those not initiated in the "d" notation, 10d3 refers to rolling 10 dice with three sides, and summing them all up. 10d3 produces a range of 10-30, with 20 being the center average.

I chose d3s since they produce a nice range with some variability but don't have extreme variability. 5d6, for example, produces a range of 6-30, which is somewhat close to 10d3's range. However the possible "burst" of a good roll means five times the low roll or twice the average for 5d6, compared to three times the low roll or 1.5 times the average in 10d3.

Once I had the damage set, I needed to guess how many hit points monsters and players should have. I started with a guess that the average combat should last five to six rounds, which with twenty points being the average round means 100-120 hit points. For the player, I split these up in health, stamina, and bonuses from the average set of equipment I'd guess one would have.

The next variable is the rate of growth as the game progresses. Magecrawl doesn't have "levels" per say, but I assign an internal level to monsters and items. Each "level" of monster and equipment is currently scheduled to increase in damage and/or stamina by about 25%. This doesn't exactly work, I realized as I wrote the previous paragraph, since the player receives a majority of their hp from their initial stamina and health. Increasing damage monsters do by 50% while increases the stamina bonus of equipment by 50% doesn't even out. Still more to hammer out apparently...

Now that I have some guesses of numbers, I coded them in and forced the map generator to produce only a single type of monster. I'm planning on playing a good number of games, tweaking the numbers so that I can "win" a decent percentage of the time but not always. Once I have that single pairing balanced, I'll continue with others until I have a good feel on the numbers that are fun....

The trouble with game balencing is that it is hard....seriously

So part of the long delay between this and the last post was a (hopefully?) well deserved break after a mammoth week of coding. However, part of it is related to my lack of progress in balancing the new monster/weapon/quality types. Trying to keep individual fights interesting while keeping ambushes from turning into two turn player-deaths and making it scale after (for now) 5 levels is hard. Like harder than writing the code that allowed it.

Part of the problem is I want to get rid of the issue I call "rounding-error bonuses". For example, let's say that having a quality weapon improves the damage by 15%. If your sword base damage is 1d4, then 15% is less than 1 point of damage. It'd have effectively no effect. One could solve this by increasing all the numbers by a factor, say 10, then the sword base damage would be 10d4 (10-40) and 15% is on average 3-4 additional damage. Getting these numbers correct, along with making them look reasonable isn't going so well. It seems weird to have the starting sword do 10d3 damage and average swings totaling 18 or more (maybe that is my experience in d&d based roguelikes and games talking).

Does anyone have a good system or set of tools for figuring this out? Bonus points if it has been proven in a shipping game or one at least farther along than magecrawl...

7DOMC - Day 7 - Monsters and thoughts on this week

With my commit here I'm done with my XML generation changes this iteration. Monsters now have different names that are level dependent, with base stats and some stats that go up each level. The "trait" system has been booted to iteration 10 due to my exhaustion in this area of magecrawl. I need to create some content for items still, item levels for each armor/weapon from 1-6 or so, but otherwise I have the base material for a sane difficulty curve.

So, if you've read this blog any this week, you probably can tell that I've been busy. With my wonderful wife coming home tomorrow and family in town next weekend, it'll probably be a few weeks before I implement anything major.

Looking back on this week, I think the fact that I announced my intentions and wrote about them every day was useful. There were days that I didn't want to program, I would have rather played TF2 or Borderlands all night. However, the idea of skipping a day's post or admitting I didn't get anything accomplished pushed me to do some programming. Once I overcame the initial hump of launching Visual Studio, I generally started to enjoy my work.

In some regards I got less done that I expected, but I think that was due to me vastly underestimating how much work some of my tasks would take. Item generation based on levels took three or four times longer that I would have initially assumed. I think it would have taken even longer if I had to break it up over a week or two's evenings.

I have 26 items left targeted for this iteration. With the possible exception to my "field" idea for improving monster AI, each of them should involve significantly less work that either the item or monster changes this week. They should be "bite sized" enough to tackle one or two after work some night.

7DOMC - Day 6 - I thought I was done with items, but I made them better.

So on day three I said I was done with the item rewrite. While this was a true statement, I realized today that there was one way I could improve items even more. The current system had each material list the base stats for a weapon/armor made of it. Something roughly:

Wood Club - 3d3  CTCost=1.0
Iron Sword - 4d6 CTCost=1.25
Cloth Armor -  +5 stamina

The issues is that I had to list out each statistic for each material. If I know that say leather is 20% more stamina than cloth (the base type), I'd need to calculate that and put both values into the XML. If I decided to increase the base later, I'd need to update all the values again.

To fix this, I now have materials list a percentage or linear amount better they are than base. For example, Wood adds 1d3 damage and leather adds 20% more stamina than base. If I update the base values, the change will propagate to all armors/weapons.

C# - The dangers of int.Parse() on non-english systems

I ran into this issue in a similar for way back in January here, but I stumbled upon in again today and I figured another post wouldn't be a bad idea.

This simple function

double ParseString(string s)
{
     return double.Parse(s);
}

is a timebomb waiting for one of your fans or customers to run into if s comes from a a source that isn't translated into the specific language. For my C/C++ fans, int.Parse() is similar to atoi() with a twist we're about to get to.

My part of the world uses a period to separate the whole from the fractional part of a number, "10.0", other parts of the world uses a comma, "10,0". C# will helpfully use the latter on systems who's language is set to be one that uses this system. This can wreck you, if for example, the string comes from xml data files that you don't want to change for each language.

There are two solutions. The simplest is to rewrite the function as follows:

double ParseString(string s)
{
     return double.Parse(s, CultureInfo.InvariantCulture);
}

That will use the "standard" english system of period for decimal seperate.

The second is one that works if you are about to do some operation that is going to call Parse a large number of times (e.g. Save/Load).

double ParseString(string s)
{
     // Save off previous culture and switch to invariant for serialization.
     CultureInfo previousCulture = Thread.CurrentThread.CurrentCulture;
     Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;

     double result = double.Parse(s, CultureInfo.InvariantCulture);

     Thread.CurrentThread.CurrentCulture = previousCulture;

     return result;
}

Just be careful that if you expect your code inside the culture setting blocks to throw an exception that you plan to survive from, to rewrite this to use finally so that it always happens.

Now I have to go check every Parse() and fix up the ones that don't happen inside a save/load call...

7DOMC - Day 5 - Monsters AI changes wrapping up

So, all the great AI work that I did a few iterations back can be distilled into this chunk of code now:

switch(typeName)
            {
                case "Bruiser":
                    tactics.Add(new DoubleSwingTactic());
                    tactics.Add(new DefaultTactic());
                    break;
                case "Healer":
                    tactics.Add(new UseFirstAidTactic());
                    tactics.Add(new MoveToWoundedAllyTactic());
                    tactics.Add(new DefaultTactic());
                    break;
                case "Ranged":
                    tactics.Add(new KeepAwayFromMeleeRangeIfAbleTactic());
                    tactics.Add(new UseSlingStoneTactic());
                    tactics.Add(new KeepAwayFromPlayerIfAbleTactic());
                    tactics.Add(new PossiblyRunFromPlayerTactic());
                    tactics.Add(new WaitTactic());
                    break;
                case "Sprinter":
                    tactics.Add(new RushTactic());
                    tactics.Add(new DefaultTactic());
                    break;
                default:
                    tactics.Add(new DefaultTactic());
                    break;
            }

I obviously plan on pulling this out into an XML list or something similar, and add more intelligent "tactics" so I can make the AI even smarter. One idea I had would have slingers who don't have a line of sight for a shot to move to one more likely. Another would have healers find bruisers and stay togeater, kinda like heavies and medics in TF2.

Now that I have a "flat" monster hierarchy, I need to write generators similar to the ones I wrote for items that'll create level dependent monsters.

7DOMC - Day 4 - Planning and Monsters

So, after the craziness of the last two days, today is a bit more subdued. Part of that is a bit of burnout and part of it is busyness at work.

Even with the work {I've/I'm going to} accomplish, my chosen tasks for iteration 9 were a bit much. I pushed back 12 minor features to another iteration, giving me 28 total. For those doing math, that doesn't exactly add up to previous posts, which is due to me splitting a few tasks up into smaller bites.

Speaking on smaller bites, I'm working on refactoring monsters completely as well. I want them to be more attribute based, like skills and items, and have more flexible behavior. Since they touch a smaller portion of the code base, I'm doing my refactoring in smaller bit sized pieces.

Update -
So I'm not done yet, but here's the jist of it.

There is a set of tactics, which are discrete single actions that could or could not be preformed. Each tactics can be asked if it could apply now and ask to do it. Then a loop like this runs:

            m_tactics.ForEach(x => x.NewTurn(this));

            foreach (IMonsterTactic tactic in m_tactics)
            {
                if (tactic.CouldUseTactic(engine, this))
                {
                    if (tactic.UseTactic(engine, this))
                        return;
                }
            }

Each monster would have a set of possible tactics, for example, my healer AI can be stated as: UseFirstAidTactic, MoveToWoundedAllyTactic, DefaultActionTactic.

Each tactic setups the required attributes on the Attributes dictionary of the monster, which are serialized, so things are remembered past save/load like the cooldown on the first aid ability.

The cute thing about this is once i code up a great new behavior, I can slip it in the monster behavior list and it'll start acting that way. Once I'm "done", I'm going to be pulling the list of behaviors a given monster has out to XML.

7DOMC - Day 3 - The item rewrite is done...finally.

So the patch I submitted was six thousand lines long, and took a grueling day of coding yesterday and half a day today cleaning up loose ends, but it's done. Much of the work involved moving a hierarchy of classes that defined weapon, armor, and consumable types into a flatter set of classes that allowed more dynamic changes.

A quick overview of the way I generate items follows:

For armor/weapons:
We start with an expected item "level", determining how "good" the item should be. We first calculate a quality grade, which falls from -2 to +4 on a normal distribution. We then determine how many "levels" we have left to choose the material. Each material, for example wood, iron, or leather, states what items can be made of it and what their properties are. Some materials have higher level modifiers, so a bronze would be lower level than say cold-forged iron. The material determines the base damage/speed/protection of the item, while the quality increases or decreases it by percentages.

For potions/scrolls/wands.

A slightly different approach is taken. An effect is first determined, using the requested level item as a range to select from. Wands cut the level in half, since they allow the effect to be produced multiple times. For any left over levels we increase either the caster level of the effect (for potions/scrolls) or the number of charges in wands.

I still need to implement the bulk of the content, for example material types for each weapon  varying over the ranges, and a ton of playtesting to find bugs and determine values for each material/quality. However, the coding for this headlining feature is done. :)

7DOMC - Day 2 - The great item rewrite

My first task today has been completed, which is moving all the skills to use the new attribute based system detailed here.

I looked at doing the same with items, when I realized that I'm going to have to rewrite pretty much my entire item hierarchy to do things right. I ripped out (with a backup copy for reference) all the item, armor, and weapon classes from the game engine, and commented out or "throw new System.NotImplementedException()"'ed all the code using them. Now I can start again from scratch and when the diffs show no commented out code, I'll be back where I started.

I very rarely try a rip out and rewrite. In general, I find a set of small incremental changes work best. However, the shear size of the item hierarchy was daunting to say the least. We'll see how this goes, I'll update later.

Update 1 (10 hours later)

So I have the weapon system up again. I've got generated weapons of various qualities and materials. The weapons damage is base speed comes from the material, with the quality affecting things as well. Descriptions are built up from the various parts, along with the names. Save/load works again for weapons. Much of the work was creating a brand new hierarchy for weapons, and the generator for materials and quality.

I'm hoping to get armor up quickly now that I have this base. Then I need to rework consumables, such as potions/scrolls/wands.

Update 2 (1 hour later)
Armor works, with quality, materials and the like. Now onto consumables.

Update 3 (1 1/2 hours later)
Obviously things are taking longer than I'd expect. I'm almost all the way through consumables but I'll have to wait for tomorrow to finish up and submit.

7DOMC - Day 1 Status

I'm winding down for the night, and I'm pretty happy what I accomplished today. Here's what I accomplished today:

• Magecrawl for Linux - This single task involved hours of work. rpath's still leave a bad taste in my mouth.
• 14 open tasks - I went from around 51 open items to 37. Some of these work items were tiny, while others involve entire features, so you can't say that I'm 27% complete however. 
• One of those open tasks required me bringing some sanity to how MagicEffectsEngine::DoEffect outputted text to the log. It look awhile, but now looking at that file no longer makes me wince.

However, other that the Linux task, I didn't touch any of my head lining features.

There is still much to be done this iteration (and this week).

Will the penguin in the room please stand up...

Magecrawl now works on Linux again. You'll need a new copy of mono (one that isn't released yet but will be before my next release), but it works!

For those who care about the bad things I had to do to make this work, read on...

So, the major issues were causing problems was the libtcod-net-unmanaged library was not working as expecting.

The first issues was both mac and linux wanted to create a library called libtcod-net-unmanaged.so. I modified the make files to create to new files and used mono's dllmap to point the right platform to the right binary.

Once I fixed that, I was getting missing symbol errors. I had forgotten that libtcod.so doesn't have the C++ symbols under Linux. I linked against libtcodxx.so, copied that in, and fixed that.

The final issue, which took three of four hours to solve, invoked this bug in cmake. I had to use an internal "don't use this, it could break on new versions" macro to work around it. The code in question is:

IF(UNIX)
     SET_TARGET_PROPERTIES(${SWIG_MODULE_${CSHARP_LIB_NAME}_REAL_NAME} PROPERTIES LINK_FLAGS "-Wl,-rpath,.")
ENDIF()

with SWIG_MODULE_${NAME}_REAL_NAME being the forbidden fruit.

7DOMC - Seven days of magecrawl coding

As I mentioned in my previous post, I have a huge list of things I'd like to accomplish in magecrawl in the short term. It just happens to be such that my wonderful wife is going to be visiting friends out of state until next Sunday. Combine this with a Monday/Tuesday holiday off work, and I have a significant amount of free time this week.

My hope is to stay productive (not just play Eve online and TF2) for most of the upcoming week. We'll see how much I get accomplished. I'll be posting updates as the week progresses.

"Iteration", I don't think that means what I think it means...

So I have some time next week to work on magecrawl, and so I'm looking at my task list and realizing I currently have a ton of things targeted for iteration 9. Ignore the bug fixing and corner cases issues, I have these high level goals currently:

• Create level dependent items and monsters
• Refactor how skills and items are handled to combine with new traits for monsters
• More effect work, including debuffs specific to various schools attack spells.
• "Field" effect to model sound/smell to make monsters seem more realistic.

In addition to this, I currently have 51 open issues targeted to iteration 9. Some of these issues are minor, but some include creating new classes of items, major new spell types, and other significant chunks of work.

I'll talk about my short term programming goals later this weekend, but the short answer is unless I seriously knock a lot of this work out soon, I'm going to have to punt on stuff to an iteration10 or have a bloated schedule.