Network compression: just say no!

There is only one way to take up less space than a cunningly compressed network packet, and that is if you don't send any data at all.

Obviously, when you send no data, things are unlikely to remain in sync from one machine to another. But you know what? Sometimes that doesn't matter.

Two rules:

  1. If it matters to gameplay, you must synchronize it
  2. Most things do not matter to gameplay

Sounds and animations rarely need to be synchronized. If the network tells you a character is moving rapidly forward, each machine can independently work out that they ought to be playing the run animation and triggering footstep sounds at regular intervals. It doesn't matter if the footsteps happen at slightly different times on each machine, so there is no need to bother synchronizing this over the network.

Case study: when you overtook another rider in MotoGP, 5% of the time they would play a "shake fist at you" animation. When you crashed, we would choose between many different animations of the rider flying through the air. This was random, and not synchronized over the network. One player might see the fist shaking while another did not, and during crashes each machine would display the rider tumbling in a different direction. We didn't have enough bandwidth to properly synchronize this stuff, but it didn't matter because the animations did not affect gameplay, so nobody noticed or cared if they were out of sync.

Similarly, in a shooter there is usually no need to synchronize the exact position of bullet hole decals, or the fact that you shot out a pane of glass, or which direction each spent cartridge particle is flying. Just send a single "I'm in ur game, firing ur gun" boolean, and each machine can figure the rest out for themselves. It doesn't matter if they each get a slightly different result, as long as the resulting parallel universes are approximately similar.

Once you get in the habit of not bothering to send information, it is amazing how much you can get away with.

Case study: in MotoGP we had advertising billboards and cones positioned around the edges of the track. If a bike collided with one of these, both object and bike would go flying. If a bike repeatedly nudged up against one while moving slowly, they could gradually move the billboard to a new location.

Most players took their racing very seriously, organizing tournaments to see who was fastest, but some griefers wanted to spoil things for everyone else. They would ride backward around the track, competing to see who could cause the biggest pile-up.

The griefers learned that if they were patient they could move several billboards out into the middle of the track, creating a barricade. The serious racers would come flying around the bend, crash into the barricade, and the dust would fly.

Thing is, this didn't actually work!

We never bothered to synchronize the position of the billboards and cones over the network. We didn't have enough bandwidth to do that.

Here is what actually happened:

  • The griefing player makes many small nudges, moving their bike slowly forward and backward.
  • We used a peer-to-peer architecture, so on the griefing machine, these movements are precisely controlled.
  • On other machines, prediction algorithms estimated where the griefer was likely to be. This was roughly right, but not exact.
  • On the griefing machine, the billboard is gradually nudged out into the middle of the track.
  • On another machine, the first nudge is slightly off to the left. This leaves the billboard in a position that is close, but not quite right. Because of that, the next nudge knocks it even further to the left. Now the griefing bike is trying to nudge the billboard, but our local version of the billboard is not where he thinks it is, so subsequent nudges have no effect. Because we do not synchronize the position of the billboard, this situation is never corrected.
  • End result: the griefer sees a barricade, but other machines do not.
  • Serious players come around the corner, then race straight past, since the barricade was never created on their machines.
  • On the griefing machine, there is a barricade. Our prediction algorithms detect a collision, so they trigger the crash animation. Dust. Sparks. Rider goes flying through the air. A moment later, a new network packet arrives. Oops! Apparently there wasn't really any collision at all. We'd better respawn the bike onto the track and carry on.

Despite millions of players putting more hours into the game than could possibly be healthy, nobody ever noticed this was not consistent.

The griefer saw a spectacular crash, and assumed everyone had been respawned after it.

The serious racers finished their serious race.

And they all lived happily ever after.

Comments (10)

  1. trayle says:

    Ok Shawn.  Enough already!  Enjoy the New Year. Ok?

    Seriously though these posts are awesome!

  2. ajmiles says:

    All this talk about MotoGP got me reminiscing about all the time I spent on this new-fangled "Xbox Live" system back in the day. I considered myself one of the better riders about at the time, but I remember one bug that now I think about it through the eyes of a game programmer, was rather strange.

    I believe it was on Barcelona that, if timed right, you could speed down the pitlane, swerve left at the last moment into one of the last pit garages and end up getting spawned over the other side of the track half a lap ahead! I can’t ever remember if this got patched or not, but quite how such a bug got written I’ve no idea, perhaps you know/remember?

    Either way though, this game was responsible for many lost days of play even with all the backwards-riding griefers that were around, an excellent title to launch XBL.

  3. Nidonocu says:

    One thing to mention would be that in Halo 3, so that apparently random things are kept in sync like choices made by the AI. The random number generator seed is shared at the start of each session once. Then every machine can correctly duplicate the same ‘random’ result.

    Maybe an article expanding on this kind of network co-op might be something interesting to cover! 🙂

  4. Ultrahead says:

    Please, someone unplug him!

  5. miguez says:

    Nidonocu brings up a very good idea, the sharing of random seeds would indeed work, and if you have a lot of random events, this would really save bandwidth if one wanted those synchronized.  If the random is triggered by an internal clock, however, this would not work, as the events would never be triggered at the same time in two or more machines, due to latency.

  6. ShawnHargreaves says:

    > I believe it was on Barcelona that, if timed right, you could speed down the pitlane, swerve left at the last moment into one of the last pit garages and end up getting spawned over the other side of the track half a lap ahead! I Lcan’t ever remember if this got patched or not, but quite how such a bug got written I’ve no idea, perhaps you know/remember?

    MotoGP, like many racing games, internally used a 2D track-relative topology to simplify computations compared to the full 3D coordinate system. Things like physics obviously used regular 3D cartesian coordinates, but for many things it can be simpler if you convert your positions to a 2D format where you have X = distance around the racetrack (0 = starting line, 1 = lapped around back to the start line again), and Y = sideways distance (0 = middle of track, -1 = left edge, 1 = right edge). That coordinate system makes it trivially easy to implement things like counting laps, figuring out who won, working out which bike is in front of the pack, dealing with steering in the AI, overtaking behaviors, etc. There’s a lot of work to write the conversion functions between regular 3D and these track-relative 2D coordinates, but once you have the 2D system, driving AI becomes much easier to write. Trouble is, we had a couple of bugs in our conversion which would sometimes choose the wrong track-relative coordinates when converting from a 3D position that was unusual and outside the normal racing area 🙂

  7. ShawnHargreaves says:

    > Enjoy the New Year. Ok?

    Hehe. I recently discovered a cool blog feature where you can write the post ahead of time, then schedule it to be published later. I wrote all this networking stuff back in early December 😉

  8. trayle says:

    >>I wrote all this networking stuff back in >>early December 😉

    Hah!  I guess computers don’t need breaks for the holidays do they?  Nice one.

  9. jwatte says:

    I think, in the end, you need to send enough data to make sure that the shared understanding of the player universe is good enough. What that means depends entirely on the context. In a racing game, you probably found a sweet spot. However, in a virtual world, that same approach would not work well.

    And, once you decide to send a packet, you should of course make sure it’s as small as possible — not doing any compression is just bad. A smaller packets means it’s less likely you overload a home network link, and for hosted games (MMOs and the likes), less network bandwidth translates into less dollars paid for hosting.

  10. jwatte says:

    > Maybe an article expanding on this kind of network co-op might be something interesting to cover! 🙂

    If you want to start learning about that, try the following link:

Skip to main content