We fulfill power fantasies

RSS Feed

Jeesgen updates and other stuff, week 10, 2018

10.3.2018 19:46:30

I've been lazy in terms of working on projects outside of work for the past month or two. I have not written proper updates for MUTA in probably over four weeks and before the ongoing week 10, I also had not touched any other non-work related programming projects.

That's not to say I haven't been programming.  Since the beginning of 2018 I have written a backend server for a game-related project (in C with MariaDB), and I have partaken in a collaborative game engine project (in C++). I just haven't touched any projects I have proper personal investment in, that's all.

But that's about to change now. I'm looking forward to getting back to MUTA, for which I will likely begin writing entity scripting next. And this week, I've again been working on Jeesgen, the static website generator I wrote for the upkeep of this site.

New features

I knew a couple of features would be needed to make the site more functional, so I implemented them this week.

Individual post pages

It is now possible to generate individual pages for each post. This allows permalinking and easier linking to articles in general. It also helps RSS generation.

Whether or not individual pages will be generated for each post is controlled by a config option in the jees_config.cfg file ('generate individual post pages = true/false').

Unlisted pages

Pages in this context are individual areas of the website which all share the same template (for example, they all have the same menu for site navigation, but different content). Previously if a new page was added to a Jeesgen project, it would automatically be added to the default navigation menu. After the update, in the original post file it is possible to specify the option 'create menu entry = true/false' to ignore the creation of a navigation menu item.

This feature is handy when you want to create a page for something that doesn't need to be accessible from everywhere. For example, I may want to create a page for MUTA, but not have it appear up top there amongst the "News", "About" and "Projects" buttons.

Upcoming features

I didn't write Jeesgen to be used much by anyone but myself, although I want it to be applicable to a multitude of different types of websites. So some fairly important features are on the hold until I get a hold of myself:

       
  • Proper date-time formatting,
  •    
  • Characters outside of the standard ascii set,
  •    
  • Install scripts for GNU/Linux and Windows,
  •    
  • Distributable binary
  •    
  • Daylight saving time

A feature I'm having problems with figuring out how to properly implement are RSS item descriptions. I like to include HTML in my posts so I in order to copy, say, the 256 first characters of a post to the description, I would have to make sure any open HTML tags are properly terminated so that they don't mess up the RSS XML. I don't see an easy way out of this, though one option could be to make the post format support Markdown, which would mean I would not have to use HTML to format posts.

Other stuff: music I've listened to recently

This isn't really related to programming or video games, but I like music, and especially I'm a big listener of electronic music. So in the name of spreading good music, below are a couple of interesting songs I've had the joy of listening to in the past few months.

Jonas Steur - Silent Waves (Youtube)

Trance

I don't know if the subgenre is Balearic or something else, but this one's a wonderfully calming track that I don't remember having heard of until recently, and neither have I paid much attention to the artist in question before.  While the calming melody feels ambient, it seems to keep on playing even after the song is over.

Anetha - Disinhibition (Youtube)

Techno

A fresh acid track that has an nicely driving bassline, giving the feeling of the beat crawling out of the listener's own head.

Spy 71 & Eva - Take Me ... Mr Love (Youtube)

Italo Disco

An italo classic and one of the songs of that era I constantly return.

Sixteen Souls - Late Night Jam (Youtube)

House

Some slightly funky, chilled house music from 1998 and nothing else.

Sophie Ellis-Bextor - Murder On The Dancefloor (Youtube)

Pop

Just pop music from the early 2000s. But with a disco/disco-house twist! And the music video is great.

C++ gripes #1: static functions as friends

20.1.2018 19:01:56

While programming as a an activity is something I, as many others, derive joy out of, there are obstacles that make it less appealing from time to time. One of those obstacles is C++. I know picking on C++ is a bit of a fashionable thing to do at the moment, but it's no wonder why: the language is easy to pick on (and it's widely used, for what ever historical reason). Well, lately I've been involved in a project utilising sepples and it's features heavily, and here's my latest gripe with the language: headers polluted with implementation details.

In C, if you want to write a static function inside your source file, unknown to the header file, and the function manipulates some data structure defined in the header, you can. For example:

foo.h:
----------------------------
struct my_struct {int x;};
----------------------------

foo.c:
----------------------------
static void
_modify_struct(struct my_struct *s)
    {s->x = 5;}
----------------------------
Well, in the world of sepples where private and public are a (rather useless) concept, it's not that simple to keep our _modify_struct() function out of the user's sight. At least for now, I could not find a way to write the equivalent of the C example up there for a C++ class in the case where the function is to modify the class' private members, not without declaring the function in the header first.

So yeah, I'm not a fan of sepples.

MUTA devlog 1: generic data files

13.1.2018 09:30:00

A devlog for what?

MUTA, an isometric 2D MMOPRG, is our main project at the moment. Therefore I thought we might go ahead and post about it's development from time to time.  So here goes: the first, official MUTA devlog! (A post to explain what the game is actually about may follow... some time in the future.)

Recent work: chat commands and file formats

Since Christmas I've mostly worked on two features for the game, apart from general stability fixes: chat commands (emotes, console commands and game master commands) and generic data files.

The emote and command stuff were simple and I added the quick versions of those earlier. The system is familiar from certain other games: a slash (/) at the beginning of a chat line indicates a command or an emote that may, depending on it's type, be processed either on the client or on the server. For example, /laugh while targeting someone may produce the text "You laugh at [TARGET]". A line beginning with a dot (.) indicates a game master command, and in the case the the full processing of the line will be done on the server, if the player has the correct privilege level - if not, the line will simply be printed as a chat message.

After adding the chat command systems I decided I should tackle generic data files - a file format that's generic enough to be used across different game systems for storing data on both, the client and the server side. The reason I picked this job next was that it would be helpful in getting many other thing done in the near future. Also it required no creative thinking, and I felt out of that juice at the moment of making the decision.

So what do you need a file format for?

In a video game there's usually plenty of data that must be stored on disk and MUTA is no exception. Some of the data is stored during runtime: for example, a client may cache data it receives from the server to reduce traffic. Such cached data could be, for example, object definitions (what sprite should a monster of id 40 use, etc.). The more traditional form of data on disk is static game data, such as assets or maps; data that only changes between game versions. I wanted to create a file format suitable for both of these use cases.

MUTA has some file types that are better specialised into their own formats: map data is one example. But a lot of files are rather similar in nature: they can, more or less, be represented as tables consisting of rows and columns. Such files in MUTA include at least:

  • The asset file, listing properties of many individual media assets - for example, what parameters is a specific texture to be loaded with,
  • Tile, creature and other object definition files that describe what sprites to use, what scripts to run and what names to give specific types of game objects.
  • It would be a lot of work to write specific file types for each one of these use cases, combined with command line or engine-integrated programs for editing them. So a generic format might be useful.

    The mdb format

    The format I wrote, called mdb, is a simple binary format. Each file has a single table (think of an SQL table), with an infinite amount of columns and rows. I already knew how some games had accomplished similar goals, and the design was certainly influenced by that.

    A row represents a single entry. For example, in a table of creatures types it would represent a single creature type.

    A column represents the value of a single field. Each column may have one of the following types (for now): int8, uint8, int16, uint16, int32, uint32 and string. There is always at least a single column: ID, which is a uint32. The ID is important because we want to be able to use that as the main reference to a specific item. For example, if I want to edit the stats of a specific creature, I must be able to find it's entry by it's ID. IDs are unique and auto incremented as a running number starting from zero when the file is created.

    As I said, the format only supports a single table per file. Why is that? Well, mostly because of time. A similar format with support for multiple tables would not be difficult to write, but due to the added complexity, it would take more time. Us being two (or at times, a little more) people working on a large project, time isn't exactly on my side. Another reason for having a single table per file is also it cuts down the amount of steps it takes to modify a single value in a file using the command line editor, which I'll get to.

    File internals

    As I said, the mdb file structure is very simple. In the header, certain fixed-size values are stored: name of the file, an automatic version, number of cols and rows, running id (all rows have an automatic uint32 id field), etc.

    After the header come columns, which store a max 32 byte string name and their type.

    Following the columns are rows. Each row is simply a sequence of raw values for each of it's fields, with the exception of strings. As a field, a string instead is a single uint32, representing an offset to a string block at the end of the file where all of the file's strings are stored. Inside that block, at the offset specified by a string field of a row, the first thing to be found is another uint32 representing the length of the string, after which comes the actual string as an array of bytes. Therefore each row physically has the same length in bytes inside the file since variable length strings are not stored in the middle of them.

    So its pretty much plain data what's inside the files and no fancy tricks (not that I would know how to pull such tricks off anyway). Additionally, because we know all of the game's data files of this type will be so small (megabytes at the largest, and rarely that), the files are always fully loaded into memory at read time - hence there are no complicated hash tables or other stuff built into the files themselves (those exist inside the in-memory version of course). If the situation comes where files become very large, well, we better write a new format or improve the existing one.

    Editor

    Of course, a binary file needs a way to edit it. I wrote a small command line program for this purpose. The program is a simple series of menus: the "new" menu asks you to fill in fields to create a new database (name, file path), where as the "open" menu let's you manipulate an existing file. The terminal dump below shows an existing file being opened.

    [owner@owner-PC mdb_edit]$ ./mdb_edit
    mdb edit v. 0
    [mdb] open    
    file path: jee.mdb
    Type 'help' for options.
    [edit db] help
    new col:   create new column
    new row:   create new row
    select:    select a row
    dump:      print out a database
    save:      save the opened file
    get field: print a field from selected row
    set field: set value of a field for selected row
    dump row:  print all the fields in the selected row

    Going forward

    There are still some things to polish up, not necessarily in the format itself but the functions used for manipulating it. When that's done, I'll get to converting some old text format files to this format.

    We are in this unfortunate situation in development at the moment where both, Lommi and I have 8 hours or more a day dedicated to doing other things than writing MUTA. But after a while, the time will come when we get some more breathing space. Until then, development should go forward slowly but steadily.

    jeesgen: a terrible static website generator

    30.12.2017 20:29:54

    The first post

    Well, this is it: the first post on the site (discluding the initial 'hello world' post). Being the first post, it be appropriate for it to explain what this site is all about. I'll briefly get that out of the way first before going into the actual topic.

    Lommi and I had been talking about creating a website for our (video game) projects for a while. During this year's Christmas holidays, I finally had time to sit down and get it done. So here we are.

    We are both programmers and hence like writing programs, some related to games, some not. So, why not have a blog go with the website where we could talk about those programs if we happened to ever have the urge to do so? Something you will likely see here in the future are technical posts about programs we've participated in writing to a lesser or greater extent. This is the first such post.

    The generator

    Alright, lets get down to the topic. What the topic is is a static website generator I wrote for the upkeep of this website recently, jeesgen. As I said, during the Christmas holidays I had a bunch of free time to get down to writing the long-dreamed-of website. So, I delved into the problem while I was away at relatives.

    The initial problem I had was that I did not know the modern tools. I'd written some static websites in the past since learning HTML at school in the early 2000s and CSS later, but that was it. How ever, that was OK for starting out, because I despise most modern websites with their all-over-the-place scripts, infinitely scrolling pages and the other fancy stuff. I support what Brian Lunduke recently said: make the web HTML again.

    So not knowing Wordpress or Javascript or what ever was not really the problem. The problem was that I wanted an easy way to update the website: things like posting news and adding new pages should be easy, preferably doable from the command line. What ever the tool for the job would be, I also hoped it to not have any external dependencies. That was a rather limiting requirement because web developers if anyone seem to have taken quite a liking to external dependencies.

    I checked out a couple of these tools quickly, mostly Jekyll and Hexo. But they seemed rather complicated to me, usually requiring at least node.js installed. They also didn't seem to easily create the types of webpages I wanted. So with that in mind, I thought: it would probably take me less time to write my own tool to do this than learn one of these existing ones anyway. Besides, writing it could be a nice holiday past time on the laptop.

    I decided to write the program in C - this would avoid any external dependencies and besides, its the perfect language for every job there is anyway. It took me an evening to get the posting feature working. Later I added pages and other stuff.

    The program is quite limited, but the idea of it, really, is to only generate the necessary parts of the site (menu buttons, posts and post pages) and embed the user's content, which may contain HTML/CSS or what ever, into that. The way the sites can be structured is not very flexible, but I don't need that: I made the tool for the type of website I would like to create, and I made it rather quickly.

    Generating a site with jeesgen

    I wanted a simple command line tool that could process text files and that's what I made. Here's the flow of creating a project and adding pages or posts to it:

         
    • Create a project directory with 'jeesgen -new my_project'
    •    
    • Modify the layout and config files inside the project. Options are included for pages, posts, page links and some other things.
    •    
    • Write a post or page into a text file.
    •    
    • Add the page or post with 'jeesgen --page' or 'jeesgen --post'
    •    
    • Generate HTML with jeesgen --generate
    And that's about precisely the workflow I wanted. In future updates I hope to add pages for specific posts, Windows compatilibity, support for umlaut characters (Finnish) and easier editing of existing content. But for now the tool should be suitable for getting the site running.

    If you are interested in the code for the tool, it may be found here. Its quite terrible, really, since it was practically just smashed in during holiday celebration off-hours, but what can you do. The program should build on GNU/Linux using gcc.

    Hello, world!

    30.12.2017 15:32:21
    Hello, world!
    Page: 0 1