A devlog for what?

MUTA, an isometric 2D MMOPRG, is our main project at the moment. Therefore I thought we might go ahead and post about it's development from time to time.  So here goes: the first, official MUTA devlog! (A post to explain what the game is actually about may follow... some time in the future.)

Recent work: chat commands and file formats

Since Christmas I've mostly worked on two features for the game, apart from general stability fixes: chat commands (emotes, console commands and game master commands) and generic data files.

The emote and command stuff were simple and I added the quick versions of those earlier. The system is familiar from certain other games: a slash (/) at the beginning of a chat line indicates a command or an emote that may, depending on it's type, be processed either on the client or on the server. For example, /laugh while targeting someone may produce the text "You laugh at [TARGET]". A line beginning with a dot (.) indicates a game master command, and in the case the the full processing of the line will be done on the server, if the player has the correct privilege level - if not, the line will simply be printed as a chat message.

After adding the chat command systems I decided I should tackle generic data files - a file format that's generic enough to be used across different game systems for storing data on both, the client and the server side. The reason I picked this job next was that it would be helpful in getting many other thing done in the near future. Also it required no creative thinking, and I felt out of that juice at the moment of making the decision.

So what do you need a file format for?

In a video game there's usually plenty of data that must be stored on disk and MUTA is no exception. Some of the data is stored during runtime: for example, a client may cache data it receives from the server to reduce traffic. Such cached data could be, for example, object definitions (what sprite should a monster of id 40 use, etc.). The more traditional form of data on disk is static game data, such as assets or maps; data that only changes between game versions. I wanted to create a file format suitable for both of these use cases.

MUTA has some file types that are better specialised into their own formats: map data is one example. But a lot of files are rather similar in nature: they can, more or less, be represented as tables consisting of rows and columns. Such files in MUTA include at least:

The mdb format

The format I wrote, called mdb, is a simple binary format. Each file has a single table (think of an SQL table), with an infinite amount of columns and rows. I already knew how some games had accomplished similar goals, and the design was certainly influenced by that.

A row represents a single entry. For example, in a table of creatures types it would represent a single creature type.

A column represents the value of a single field. Each column may have one of the following types (for now): int8, uint8, int16, uint16, int32, uint32 and string. There is always at least a single column: ID, which is a uint32. The ID is important because we want to be able to use that as the main reference to a specific item. For example, if I want to edit the stats of a specific creature, I must be able to find it's entry by it's ID. IDs are unique and auto incremented as a running number starting from zero when the file is created.

As I said, the format only supports a single table per file. Why is that? Well, mostly because of time. A similar format with support for multiple tables would not be difficult to write, but due to the added complexity, it would take more time. Us being two (or at times, a little more) people working on a large project, time isn't exactly on my side. Another reason for having a single table per file is also it cuts down the amount of steps it takes to modify a single value in a file using the command line editor, which I'll get to.

File internals

As I said, the mdb file structure is very simple. In the header, certain fixed-size values are stored: name of the file, an automatic version, number of cols and rows, running id (all rows have an automatic uint32 id field), etc.

After the header come columns, which store a max 32 byte string name and their type.

Following the columns are rows. Each row is simply a sequence of raw values for each of it's fields, with the exception of strings. As a field, a string instead is a single uint32, representing an offset to a string block at the end of the file where all of the file's strings are stored. Inside that block, at the offset specified by a string field of a row, the first thing to be found is another uint32 representing the length of the string, after which comes the actual string as an array of bytes. Therefore each row physically has the same length in bytes inside the file since variable length strings are not stored in the middle of them.

So its pretty much plain data what's inside the files and no fancy tricks (not that I would know how to pull such tricks off anyway). Additionally, because we know all of the game's data files of this type will be so small (megabytes at the largest, and rarely that), the files are always fully loaded into memory at read time - hence there are no complicated hash tables or other stuff built into the files themselves (those exist inside the in-memory version of course). If the situation comes where files become very large, well, we better write a new format or improve the existing one.

Editor

Of course, a binary file needs a way to edit it. I wrote a small command line program for this purpose. The program is a simple series of menus: the "new" menu asks you to fill in fields to create a new database (name, file path), where as the "open" menu let's you manipulate an existing file. The terminal dump below shows an existing file being opened.

Going forward

There are still some things to polish up, not necessarily in the format itself but the functions used for manipulating it. When that's done, I'll get to converting some old text format files to this format.

We are in this unfortunate situation in development at the moment where both, Lommi and I have 8 hours or more a day dedicated to doing other things than writing MUTA. But after a while, the time will come when we get some more breathing space. Until then, development should go forward slowly but steadily.