A devlog for what?

MUTA, an isometric 2D MMOPRG, is our main project at the moment. Therefore I thought we might go ahead and post about it's development from time to time.  So here goes: the first, official MUTA devlog! (A post to explain what the game is actually about may follow... some time in the future.)

Recent work: chat commands and file formats

Since Christmas I've mostly worked on two features for the game, apart from general stability fixes: chat commands (emotes, console commands and game master commands) and generic data files.

The emote and command stuff were simple and I added the quick versions of those earlier. The system is familiar from certain other games: a slash (/) at the beginning of a chat line indicates a command or an emote that may, depending on it's type, be processed either on the client or on the server. For example, /laugh while targeting someone may produce the text "You laugh at [TARGET]". A line beginning with a dot (.) indicates a game master command, and in the case the the full processing of the line will be done on the server, if the player has the correct privilege level - if not, the line will simply be printed as a chat message.

After adding the chat command systems I decided I should tackle generic data files - a file format that's generic enough to be used across different game systems for storing data on both, the client and the server side. The reason I picked this job next was that it would be helpful in getting many other thing done in the near future. Also it required no creative thinking, and I felt out of that juice at the moment of making the decision.

So what do you need a file format for?

In a video game there's usually plenty of data that must be stored on disk and MUTA is no exception. Some of the data is stored during runtime: for example, a client may cache data it receives from the server to reduce traffic. Such cached data could be, for example, object definitions (what sprite should a monster of id 40 use, etc.). The more traditional form of data on disk is static game data, such as assets or maps; data that only changes between game versions. I wanted to create a file format suitable for both of these use cases.

MUTA has some file types that are better specialised into their own formats: map data is one example. But a lot of files are rather similar in nature: they can, more or less, be represented as tables consisting of rows and columns. Such files in MUTA include at least:

The mdb format

The format I wrote, called mdb, is a simple binary format. Each file has a single table (think of an SQL table), with an infinite amount of columns and rows. I already knew how some games had accomplished similar goals, and the design was certainly influenced by that.

A row represents a single entry. For example, in a table of creatures types it would represent a single creature type.

A column represents the value of a single field. Each column may have one of the following types (for now): int8, uint8, int16, uint16, int32, uint32 and string. There is always at least a single column: ID, which is a uint32. The ID is important because we want to be able to use that as the main reference to a specific item. For example, if I want to edit the stats of a specific creature, I must be able to find it's entry by it's ID. IDs are unique and auto incremented as a running number starting from zero when the file is created.

As I said, the format only supports a single table per file. Why is that? Well, mostly because of time. A similar format with support for multiple tables would not be difficult to write, but due to the added complexity, it would take more time. Us being two (or at times, a little more) people working on a large project, time isn't exactly on my side. Another reason for having a single table per file is also it cuts down the amount of steps it takes to modify a single value in a file using the command line editor, which I'll get to.

File internals

As I said, the mdb file structure is very simple. In the header, certain fixed-size values are stored: name of the file, an automatic version, number of cols and rows, running id (all rows have an automatic uint32 id field), etc.

After the header come columns, which store a max 32 byte string name and their type.

Following the columns are rows. Each row is simply a sequence of raw values for each of it's fields, with the exception of strings. As a field, a string instead is a single uint32, representing an offset to a string block at the end of the file where all of the file's strings are stored. Inside that block, at the offset specified by a string field of a row, the first thing to be found is another uint32 representing the length of the string, after which comes the actual string as an array of bytes. Therefore each row physically has the same length in bytes inside the file since variable length strings are not stored in the middle of them.

So its pretty much plain data what's inside the files and no fancy tricks (not that I would know how to pull such tricks off anyway). Additionally, because we know all of the game's data files of this type will be so small (megabytes at the largest, and rarely that), the files are always fully loaded into memory at read time - hence there are no complicated hash tables or other stuff built into the files themselves (those exist inside the in-memory version of course). If the situation comes where files become very large, well, we better write a new format or improve the existing one.

Editor

Of course, a binary file needs a way to edit it. I wrote a small command line program for this purpose. The program is a simple series of menus: the "new" menu asks you to fill in fields to create a new database (name, file path), where as the "open" menu let's you manipulate an existing file. The terminal dump below shows an existing file being opened.

Going forward

There are still some things to polish up, not necessarily in the format itself but the functions used for manipulating it. When that's done, I'll get to converting some old text format files to this format.

We are in this unfortunate situation in development at the moment where both, Lommi and I have 8 hours or more a day dedicated to doing other things than writing MUTA. But after a while, the time will come when we get some more breathing space. Until then, development should go forward slowly but steadily.

The first post

Well, this is it: the first post on the site (discluding the initial 'hello world' post). Being the first post, it be appropriate for it to explain what this site is all about. I'll briefly get that out of the way first before going into the actual topic.

Lommi and I had been talking about creating a website for our (video game) projects for a while. During this year's Christmas holidays, I finally had time to sit down and get it done. So here we are.

We are both programmers and hence like writing programs, some related to games, some not. So, why not have a blog go with the website where we could talk about those programs if we happened to ever have the urge to do so? Something you will likely see here in the future are technical posts about programs we've participated in writing to a lesser or greater extent. This is the first such post.

The generator

Alright, lets get down to the topic. What the topic is is a static website generator I wrote for the upkeep of this website recently, jeesgen. As I said, during the Christmas holidays I had a bunch of free time to get down to writing the long-dreamed-of website. So, I delved into the problem while I was away at relatives.

The initial problem I had was that I did not know the modern tools. I'd written some static websites in the past since learning HTML at school in the early 2000s and CSS later, but that was it. How ever, that was OK for starting out, because I despise most modern websites with their all-over-the-place scripts, infinitely scrolling pages and the other fancy stuff. I support what Brian Lunduke recently said: make the web HTML again.

So not knowing Wordpress or Javascript or what ever was not really the problem. The problem was that I wanted an easy way to update the website: things like posting news and adding new pages should be easy, preferably doable from the command line. What ever the tool for the job would be, I also hoped it to not have any external dependencies. That was a rather limiting requirement because web developers if anyone seem to have taken quite a liking to external dependencies.

I checked out a couple of these tools quickly, mostly Jekyll and Hexo. But they seemed rather complicated to me, usually requiring at least node.js installed. They also didn't seem to easily create the types of webpages I wanted. So with that in mind, I thought: it would probably take me less time to write my own tool to do this than learn one of these existing ones anyway. Besides, writing it could be a nice holiday past time on the laptop.

I decided to write the program in C - this would avoid any external dependencies and besides, its the perfect language for every job there is anyway. It took me an evening to get the posting feature working. Later I added pages and other stuff.

The program is quite limited, but the idea of it, really, is to only generate the necessary parts of the site (menu buttons, posts and post pages) and embed the user's content, which may contain HTML/CSS or what ever, into that. The way the sites can be structured is not very flexible, but I don't need that: I made the tool for the type of website I would like to create, and I made it rather quickly.

Generating a site with jeesgen

I wanted a simple command line tool that could process text files and that's what I made. Here's the flow of creating a project and adding pages or posts to it:

   
• Create a project directory with 'jeesgen -new my_project'
   
• Modify the layout and config files inside the project. Options are included for pages, posts, page links and some other things.
   
• Write a post or page into a text file.
   
• Add the page or post with 'jeesgen --page' or 'jeesgen --post'
   
• Generate HTML with jeesgen --generate

If you are interested in the code for the tool, it may be found here. Its quite terrible, really, since it was practically just smashed in during holiday celebration off-hours, but what can you do. The program should build on GNU/Linux using gcc.