Serve statics from somewhere else

In the beginning, letting your Rails setup serve everything is very easy. But as your game grows in popularity, you’ll notice that your app spends a lot of time serving up things that never change.

The first thing we did was separate out all the static images onto a separate server running nginx. Nginx is notoriously fast at serving static content and it did a good job for us. This left our Rails instances to serve only game-related activities.

Another useful bit was using a plugin called bundle_fu. Bundle_fu takes all the javascrips and stylesheets you use in your app and creates a single file for the browser to download. So, instead of the browser having to make 6 connections to get all those pieces, it makes 1. This functionality is built into Rails since version 2.0.

Optimize your environment

Initially, we ran Apache proxying to 5 to 10 mongrel servers running Rails. Then we switched to Nginx/mongrels which gave us a bit of speed and a bit of memory back, after doing some more searching we eventually switched entirely to the Litespeed webserver with no mongrels. Litespeed is an awesome product and we have had rock solid performance with it for the last year. Although, all my latest Rails deployments have been with Apache & mod_rails (Passenger) running with the Ruby Enterprise Edition.

One big upside to Litespeed and Passenger is that you don’t have to manage mongrels anymore and restarting the website is super fast. Also, Ruby Enterprise Edition (REE) can reduce your memory requirements up to 30%.

Memcached

I think the biggest anecdotal speed increase we got was from integrating support for memcached. It was as simple as adding the acts_as_cached plugin and marking a few models to be cached. From there, we began optimizing which models were cached for the best performance.

Watch for slow queries

Edit your mysql.cnf file to start logging slow queries and then watch that log like a hawk. When you see a query that isn’t using an index, make sure you add the appropriate index to get that query to be fast again. This won’t help much if the query only runs once a day, but if it runs every few seconds — you’ll notice the difference.

Throttling the game

Early on when there were only a few hundred people in our game we added a bunch of notifications to our chat system so that people would know that others were there and something was happening. Crank that up to several thousand people and those notifications are nothing but annoying. So, we added code that checked for the number of currently online people and throttled those informative messages when things got busy and brought them back when things calmed down again. This made a big improvement in the quality of play during busy times.

These are just a few of the things we tried, but they were the most successful at improving the performance of our game.

If you’d like to see it, browse over to Tooth & Claw

The problem is that you have an object with a ton of properties that are incredibly similar and they describe the object in a yes|no fashion. In my case, players can do 24 types of research and the state of the player is “yes, I have done that particular research” or “no, I have not done that research”.

One solution is to make a table with a column that associates with the player id and a boolean column for every type of research that you have. Now if you have 24 types of research your table is 25 columns big. This can get out of hand pretty quickly. The table becomes hard to read and you have to use different code (or procedurally dynamic code) to set individual columns.

Another solution is to add a column to your player definition table and make it type INT UNSIGNED. Then you let your code efficiently handle interpreting the integer as the player’s research definition through bit masking. Here’s how it works.

The maximum value of an unsigned INT in MySQL is 4294967295. In binary this number looks like 11111111111111111111111111111111. That is 32 1’s in a row. Each of those digits can describe a research type as ‘have’ (it is a 1) or ‘have not’ (it is a 0). Now in a global file for your code you need to define each research type as a number that is a power of 2. It would look something like this in PHP:

$g_shield_research = 1;              // in binary 001
$g_armor_piercing_research = 2;  // in binary 010
$g_mining_research = 4;             // in binary 100

Now if you want to know whether you have a particular research you would perform a bitmask operation on the integer you retrieve from your database using the & operator.

// will mask players research and return true if the mining bit is set to 1
if ($element->research & $g_mining_research)

The bit masking procedure is extremely efficient and fast and you can see how it compresses all the research information into the size of an integer. Also, if you want to know everything about a player’s research you only have to retrieve a single integer from the database.

Assigning research to a player is also very easy. Simply bitwise OR the current research integer with the set bitmask using the | operator:

$newPlayerResearch = $element->research | $g_shield_research;

You can technically add the two numbers to get the same result, but this is unsafe because if you add the research when it is already there it will throw everything off.

There are some pitfalls to using this trick. While it is easy to add another type of research just by assigning its mask to the next highest power of 2, you are limited to 32 total research types. One way to get around this is to make the column type BIGINT which would give you 64 bits to work with, but at the end of the day you are still limited. Also, once a game starts the research you choose for that bit position is pretty much stuck there unless you want to do some math maintenance.

While this trick will tend to make your code more readable because database statements won’t be as long, your database entry will not be human readable so it could slow down your debugging efforts.

So there you have it. A very powerful tool if used wisely. Please design well before you start writing code. It makes life easier.

At the time of this writing, Twitter is having some scaling problems. Twitter is using Ruby on Rails for their service, and a lot of their problems are being blamed on the fact that Rails apparently does not scale well to having millions of users. This has led a lot of new and prospective developers to choose something other than Rails, based on the statement that “Rails doesn’t scale”. But the reality is, Twitter is huge. And your game is not. There are users on Twitter with over 1500 followers – chances are, your game doesn’t even have that many users if you just launched it.

Scalability isn’t something you should be worrying about when you’re first starting out. You need to attract users, and keep them. As long as your game can handle 100+ users at once, you don’t need to worry about scaling. And trust me, most things can handle over a hundred users at once.

Something like 3/4’s of game projects that are started never see the light of day. When you’re first building your game, you need to focus on getting something out the door and playable – not making sure it’s absolutely perfect. That’s what the testing period and your initial playerbase will help you get figured out. Most of the other aspects of building your game aren’t a big deal – just get the original out there! Once you have something that people are playing, you’ll be able to evalulate it better and see what areas you can improve in – and hey, if your playerbase suddenly explodes, you’ll get to learn about how to make your game scale, too. But please, please – stop treating scalability like it’s your biggest concern.