Building Browsergames

Saving Database Space through Bit-masking

This is a trick you can use to increase the efficiency and readability of your project. It is an argument for good up front design as utilizing this is only plausible when you take the time and effort at the beginning. The following is a real world example from my game TerraTanks.

The problem is that you have an object with a ton of properties that are incredibly similar and they describe the object in a yes|no fashion. In my case, players can do 24 types of research and the state of the player is “yes, I have done that particular research” or “no, I have not done that research”.

One solution is to make a table with a column that associates with the player id and a boolean column for every type of research that you have. Now if you have 24 types of research your table is 25 columns big. This can get out of hand pretty quickly. The table becomes hard to read and you have to use different code (or procedurally dynamic code) to set individual columns.

Another solution is to add a column to your player definition table and make it type INT UNSIGNED. Then you let your code efficiently handle interpreting the integer as the player’s research definition through bit masking. Here’s how it works.

The maximum value of an unsigned INT in MySQL is 4294967295. In binary this number looks like 11111111111111111111111111111111. That is 32 1’s in a row. Each of those digits can describe a research type as ‘have’ (it is a 1) or ‘have not’ (it is a 0). Now in a global file for your code you need to define each research type as a number that is a power of 2. It would look something like this in PHP:

$g_shield_research = 1;              // in binary 001
$g_armor_piercing_research = 2;  // in binary 010
$g_mining_research = 4;             // in binary 100

Now if you want to know whether you have a particular research you would perform a bitmask operation on the integer you retrieve from your database using the & operator.

// will mask players research and return true if the mining bit is set to 1
if ($element->research & $g_mining_research)

The bit masking procedure is extremely efficient and fast and you can see how it compresses all the research information into the size of an integer. Also, if you want to know everything about a player’s research you only have to retrieve a single integer from the database.

Assigning research to a player is also very easy. Simply bitwise OR the current research integer with the set bitmask using the | operator:

$newPlayerResearch = $element->research | $g_shield_research;

You can technically add the two numbers to get the same result, but this is unsafe because if you add the research when it is already there it will throw everything off.

There are some pitfalls to using this trick. While it is easy to add another type of research just by assigning its mask to the next highest power of 2, you are limited to 32 total research types. One way to get around this is to make the column type BIGINT which would give you 64 bits to work with, but at the end of the day you are still limited. Also, once a game starts the research you choose for that bit position is pretty much stuck there unless you want to do some math maintenance.

While this trick will tend to make your code more readable because database statements won’t be as long, your database entry will not be human readable so it could slow down your debugging efforts.

So there you have it. A very powerful tool if used wisely. Please design well before you start writing code. It makes life easier.

Scalability is not your biggest concern, so stop acting like it is

I can’t count the number of times that I’ve heard things like ‘I was thinking of doing <insert approach here>, but heard that it doesn’t scale well once you hit more than about 1000 users’. It happens a lot. And you know why it happens? Because people who are new to developing think that how well their system scales is a big deal. I have a secret piece of information for you: scalability’s not the big deal you think it is.

At the time of this writing, Twitter is having some scaling problems. Twitter is using Ruby on Rails for their service, and a lot of their problems are being blamed on the fact that Rails apparently does not scale well to having millions of users. This has led a lot of new and prospective developers to choose something other than Rails, based on the statement that “Rails doesn’t scale”. But the reality is, Twitter is huge. And your game is not. There are users on Twitter with over 1500 followers - chances are, your game doesn’t even have that many users if you just launched it.

Scalability isn’t something you should be worrying about when you’re first starting out. You need to attract users, and keep them. As long as your game can handle 100+ users at once, you don’t need to worry about scaling. And trust me, most things can handle over a hundred users at once.

Something like 3/4’s of game projects that are started never see the light of day. When you’re first building your game, you need to focus on getting something out the door and playable - not making sure it’s absolutely perfect. That’s what the testing period and your initial playerbase will help you get figured out. Most of the other aspects of building your game aren’t a big deal - just get the original out there! Once you have something that people are playing, you’ll be able to evalulate it better and see what areas you can improve in - and hey, if your playerbase suddenly explodes, you’ll get to learn about how to make your game scale, too. But please, please - stop treating scalability like it’s your biggest concern.