A couple of months ago a friend of mine informed me that my school, Kajaani University of Applied Sciences (KAMK), was going send a delegation to the state of Andhra Pradesh, India, to teach game development for two months. The event would be known as IGDC, short for Indian Game Development Challenge. KAMK was recruiting for the job so I applied and was accepted. I left Finland at the beginning of June and came back home just a day ago, 30th July. I will now attempt to describe the experience in words.

Note that this text is written from my personal perspective and does not represent the views of KAMK nor anyone else who isn't me.

Preparing and leaving for the trip

At some point in time KAMK, my school, had come in contact with APSSDC, the Andhra Pradesh State Skill Development Corporation. They had agreed on KAMK providing the programme for a two month summer course on video game development. Initially the idea had been about bringing Indian students to Finland for the summer, but this was soon diverted so that instead of sending Indians to Finland, KAMK would be sending Finns to India as teachers and organizers. A friend of mine asked me to apply to be part of the delegation going to India and I was accepted. The job was to teach and coach students at the SRM University, Amaravathi.

I don't remember when exactly I got to know I was going to India, but at most I believe it was two months or so before the actual date of leaving. I was working 8 hours a day on an engineering project at that point, so I had little time left for preparing course material or the like. The most work I got to do regarding the India project was when me and my colleague were asked to give an online presentation to some of the Indian students along with a questions and answers session. We staid at work until 9 PM making a PowerPoint because we were told the presentation would need to happen the next day at 7:30 AM. I have to say at this point that little to no time for preparation was sort of a defining feature of this trip. Anyway, the Q&A session left me with one idea on the top of the pile: I would have to work on my understanding of Indian English accents. However, it was also nice to get to know how excited some of the students already seemed.

During the same week we did the Q&A session, three people from India came to visit us in Kajaani, Finland. Two of them were from the APSSDC and one represented SRM University, the school where the course would be held. The Indians were taken hiking as well as to the sauna. Later I heard they were also taken to to see a Finnish dump pit (amongst various other things) which I thought was funny as hell. This was my first contact with Indian people, but also the first time I got to talk to our Finnish project coordinator, an all-around nice guy who would play a very important role during our stay in AP.

Finally at the beginning of June 2018 it was time to leave. I left with one other person only instead of the whole team due to various reasons I will not go into now. Our coordinator, mentioned in the above paragraph, was already waiting for us at SRM University Amaravathi, the campus we would spend the next two months at.

Arrival

We arrived near the weekend and soon met with our own man on the spot, as well as the faculty of the school. We got to see the equipment, which seemed satisfactory, but more computers would be needed - we were going to have nearly 500 students (the APSSDC took care of this well later). The lack of equipment at this point (I assume) had to do with the fact the whole school was still being built. In fact, being built is what was happening to the whole city of Amaravathi 24/7 around the university. The story goes, a couple of years ago the state had broken up into two states and the old capital, Hyderabad, staid in the other state (Telangana), where as Andhra Pradesh was going to build it's own capital from scratch. Indeed, every hour of the day the construction of the school and the city were on-going (which did not help our night sleep at all, by the way).

For our first Sunday (the working week is 6 days in India) one of our local contacts, did something very nice and took us to see his home as well as some other places nearby. That was a gesture we very much appreciated as we got a glimpse of what the country looked like outside of the borders of the campus. The area we were staying at was quite rural, so traveling on the muddy roads took a lot of time. We went to a beach as well and were probably the only people there with nothing but boxers on - I learned later Indians swim in full clothing. Of course we also went to a small temple and ate well.

The first week went on fast. I taught the students some basic concepts of game programming, like what is a program main loop, what tools can you use, etc. as well as practical game programming in C with SDL2. We also talked about version control using Git. A lot of information had to be packed into a very short period of time, and I had not had the time to really prepare any materials at home. I still regret the lack of prepared material, but many people seemed to already catch on and start working on their games. I spent a lot of time in the evenings later preparing course materials for the future since I had it fresh on my mind what was going to be needed.

A week later the rest of our team of 8 Finns arrived. Something that made the first weeks rough for us all was that students did not arrive in one bunch but at the rate of about 40 per day. The total amount at the end was about 480. This meant many things had to be taught again and again, often separately to only a few students at a time. Luckily Indians are (from my limited experience at least) social people and often the students would by themselves inform any new arrivals about various important things.

Course structure

Why would a group of Finnish people be sent to India to teach game development? My understanding is that our job was to introduce the students (and the faculty) to a more practical way of learning, since Indian education is still quite conservative, often (again, to my understanding at least) being about listening to the teacher babble while taking notes. Well, our course was certainly all about practice and not about listening to the teacher babble.

We did some lecturing at the beginning in the form if intensive courses, but just like KAMK game dev summer courses in Finland, most of the time the students worked on their game projects in teams of 4-10 using the tools they wanted. Us teachers, we were simply there available for any questions (or queries, as Indians say) they came up with, occasionally roaming around and checking out the progress being made. People caught onto this model surprisingly quickly, although I have some doubts that there were people even at the end of the course who would have preferred a more theoretical approach - different styles suit different people.

Cultural differences

For someone like me who's never been outside of Europe, seeing even a limited part of India is an eye-opening experience. Just like the EU, India consists of various states with different cultures and languages, but being an ignoramus I had little prior knowledge about how the nation was actually organized. Now I know a little more, but just a little - we only staid in one state (AP), mostly at a university campus, apart from a night or two in Delhi (and that in the near vicinity of the airport - the furthest we got out of the hotel was the liquor store).

Still regarding culture, it warms my heart to see a nation not yet completely taken over by American pop culture (Finland is just about a lost cause at this). Of course many (especially men) wear western clothing and some people know the big Hollywood or pop EDM hits, etc. But at least in the rural areas it seems that, despite the massive presence of American companies such as Coca Cola, Pepsi and the like, people still have their own style of clothing, music, films and customs. That is refreshing and gives some hope for the future of this planet to me.

To just shortly list some differences between Finnish and Indian culture: friendliness (Finnish people don't smile at strangers that much); respect (a nobody like me will get called sir or madam in India, especially wile wearing a blazer); respect for other's time (Indians have little to no sense of punctuality); coffee (Indians have to forcefully put milk and sugar into it). I also cannot help but quote a colleague of mine: how do Indians ever get anywhere by walking so slow. And yet, we all share the same basic needs and understanding of human ethics.

Improvements for the future

This summer course was the first in what is hoped to be many in the near future. Because of that our team had to learn many things in practice. Here are some of those things.

Don't make the course a competition

Since the beginning it was planned that we would choose one team from the course who would be sent to the Slush business event in Helsinki, Finland to pitch their project and company. Immediately upon hearing about this I felt it was a bad idea as it would pitch students against one another. And I feel like that exactly happened, especially towards the end. Games are a business but at least in Finland companies tend to collaborate and share information - ruthless competition isn't that big a part of the picture, or at least that's the feeling I get. Thinking of your co-learner or colleague as a competitor only creates bad blood and burns bridges.

Don't overpromise

We tried hard not to overpromise, but sometimes you should just keep your mouth completely shut instead of telling someone of the possibility of something happening. For example, we were expecting three companies from Kajaani to visit the university, but we should not have spoken about it before the companies confirmed that they were indeed coming. In the end, only one of the companies paid a visit (my greatest thanks go to Virtual Frontiers for this by the way).

Let all students show off their progress

We had nearly 500 people working on various game projects for two full months. That's a lot of working hours, especially considering the working week is 6 days in India and the working day seems to be of a varying length upwards from 8 hours. A massive effort from some very dedicated people to say the least.

All project courses at KAMK end in a post mortem session where students get to dissect everything that went right or wrong during the project period. This way students get to learn from others' mistakes and successes, but at the same time, a post mortem acts as a public revelation of a student project to the public - finally you get to show off the hard work you've done, and usually you also get some useful commentary! The post mortem event is one of the highlights of a course.

Because of scheduling problems, our own mistakes in management, travel timings, stress and various other excuses, we forgot about the post mortems at this event. That was a mistake that I feel made many students feel disappointed, to feel as if their work was not appreciated. That still bothers me. When you've made something, you want to tell the world, and you should be given the chance to do that. And of course, we would have wanted to see those games presented, too.

Keep on pitching

When a project course begins at KAMK, students must first pitch their projects to teachers and get them accepted. We did the same thing at the IGDC, but also kept on holding practice pitching sessions every Friday for the full duration of the course. That was great and should be continued - it was really motivating to see the improvements in the presentations made by the students. Some definite performers there.

Regards

All in all, for me the overwhelming feeling left of this trip is positivity. My heartfelt thanks go to the motivated and friendly students, to the helpful faculty of the organizations involved as well as to the various other great people I (we) met on the way.

The greatest motivator for me was always the progress and enthusiasm of the students who I would thank all personally did I have the time. Here's to hoping for more collaboration between Indian and Finnish educational agencies, and for more events of this kind in the future.

My most recent workings on MUTA have been about adding ways to implement game content. I have now added the first version of the server side scripting language, Lua. In relation to this work, I've had to implement multiple data file formats so that we know thing slike which scripts to load and which game object uses which script.

File formats for defining game objects

The first thing I wanted to try scripting with were creatures. But before I could script creatures, we had to be able to define them. For that a file format was required. I had used ini-type files so far for quick debug purposes, but I decided it was time to implement a better format.

I've written earlier about the binary mdb file format I wrote for use in the game - we use it for art assets. How ever, while debugging and trying out new stuff, binary files with special editors (mdb has a command line editor in which you give commands like "set column") are a little cumbersome. Text files on the other hand are easy to modify on the go. So a text-based format seemed logical for something like creature definitions that - entries that would change frequently.

Definitions like these aren't complex to represent. An ini file would not do it, but something very close to it would. So here's how our creature definition files look like now:

A similar format is handy for a lot of things: creature, dynamic object, static object and player race definitions to name some. So there's a generalized parser function:

As the function parses the file given in parameter fp, the on_def callback is called when a new definition beginning is found, for example "creature: human_male_1". The callback on_opt is called when an ini-like option is encountered. Ctx is an optional pointer to a caller-defined context. The option name and the value are passed as strings to the callbacks. A missing feature for now are multi-line values, which we might need at some point for longer pieces of text like object descriptions. It's not fancy (actually it's almost the same as .desktop files on Linux or many other similar formats), but these files are both, easy to change on the go and human readable.

Lua scripting creatures

The addition of a scripting language is something I've been meaning to get to for a longer while, but everytime I've meant to get to it, I've found some other feature missing and wandered off. A couple of weeks back I finally got my hands dirty with Lua.

I had not used Lua earlier so it took a little bit of reading the reference manual and the Programming in Lua text found online to figure out how to use the thing. Lua is of course one of, if not
the
most popular embedded scripting language, so the process was fairly simple - if it wasn't, I doubt the language would have such a following.

The first thing I decided to apply scripting for were creatures. Here's how a creature script might look like at the moment:

The Init() function is called when the script it attached to an entity. The Tick function is called every frame if it exists. As more use cases come up, more functions will probably be added. All of the functions will probably be optional, too, so if an entity doesn't require a script function call every frame, the Tick function for example could be omitted.

The MUTA Lua-side API will probably consist of mostly C functions bound to Lua. In the example above for example, creature_walk is a C function binding. Functions like these operate directly on raw entity pointers (Lua's 'user data' type). I don't know yet if that's a good idea, but it's certainly easy. One thing that must be kept in mind is that since our world will be divided into multiple 'worldd' processes, each simulating a part of the map, the scripts must be movable from one node to another and back again.

Defining scripts

All scripts to be loaded at server start-up must be defined in an ini file. Each line of the file defines the name and the file path of a script, for instance, "test = muta-data/server/scripts/test.lua". This associates an easy-to-remember string id with the script. The given name can then be used in game object definition files to refer to the script.

Script hot reloading

The nice thing about scripts is that you can modify them as the program is running. The way this functionality works in MUTA is that a character with the game master status calls a chat command with the script's name.

When a GM types in, '.reload_script test', a command is sent to every worldd process connected to the master server to reload the script 'test'.  Once the worldd's receive the command, they will look up the script, reload it's Lua code and re-register it's functions in Lua's own registry, then find every entity in the world using the script and reload it for them. That's quite nice for fast iteration!

Packetwriter improvements

Elsewhere, Lommi has been working on improving MUTA Packetwriter, the program we use to define and generate code for network packets. The program works so that every packet in the protocol is defined in a .mp file, which is passed as a parameter to the command-line packetwriter program. The program reads the file and forms the correct structs and inline functions for handling the defined packets, writing them into a C header file.  While the changes aren't upstream yet, there's a bunch of nice stuff incoming.

The improvements include packet definition formatting that isn't as strict as it used to be. It used to be that a white space or linebreak in the wrong place could break the generation but the program would not tell you want went wrong, instead just generating broken code.

You're also now allowed to do arithmetic when defining packet specific size limits.  This is a handy feature for packets that contain variable size elements such as strings.

Support for arrays of packed structs is also on it's way. This is a feature I've been wanting a lot because it's damn handy. For example when sending an account's character list, instead of being able to send an array of structs I've had to create an individual array for every parameter of every character. This has resulted in a lot of friction because populating multiple arrays is obviously more complex than populating a single array, and since our way of using packets is fairly low level, there have been bugs with uninitialized variables and such that took a while to debug.

Up next

I'm currently working on improving creatures. I'm not quite sure what exactly I'll be doing next, but it will be about making gameplay implementation easier, be it more Lua API functionality or actual game object interaction between the client and the server. Better get coding right now!

This is really just a heads up for anyone who missed it: this year's GDC featured a new post mortem session for the now 20 year old Ultima Online, and the video is already available for free here.

UO remains a source of interesting MMO history and design topics to this day. Although the session featured many of the same talking points as the post mortem in 2012, there was a little bit of new stuff there, too. For example, this time Richard Garriott himself took part, and Rich Vogel talked about the invention of game time cards.

I only wish I could find an interview with Rick Delashmit, the original main programmer of the game. It would be interesting to know for example how they stored world state information and account data on disk back in the day!

Alright, here's another post about MUTA, our MMORPG project. I didn't work much on the game for nearly two months since the new year holidays, but I've since been picking up the pace, working the hours I could afford. Unfortunately, it's not that much, but evenings are still something as are weekends. The days are also getting longer and longer, and I've noticed in the past my productivity drops drastically in the middle of the winter when there's only a couple of hours of daylight. Thank goodness spring seems to be making it's way swiftly!

Since Christmas, I believe I alone have touched the codebase. This has a clear advantage: there's no way to get merge conflicts if you only merge from one branch. So, I have been cleaning up here and there, including in files I didn't previously touch personally (although I didn't do anything drastic in the case of such files.), reformatting, clarifying naming conventions, etc. I also implemented asynchronous asset loading proper and started adding support for creatures, but more on that later.

Getting rid of unneeded code

Something I have been wanting to do is getting rid of code that isn't needed. Obviously, the smaller, the better when it comes to codebases - I think we can all agree on that. And we did have some code that was practically duplicate.

For example, we originally had a hashtable implementation that preserved memory addresses and another hashtable that didn't. In the case of the first container, instead of increasing the amount of buckets when more space was needed, a new bucket was created and added to a linked list of buckets at index N. Now, that's not great algorithmically because obviously search time increases linearly with the amount of items in a bucket. But I thought it would be handy when you wanted to store items instead of pointers inside a hashtable, and also wanted pointers to those items to remain valid until the bitter end, even if the table was resized.

Well, turns out the first hash table was only used in one file, in our asset system, and even there the only reason for it's use was that this table was implemented first, back when the other hashtable had not been written yet.

So I got rid of the memory address preserving hashtable.

Less macros

Our containers, including the hashtables, have previously been implemented as C++-template-like C preprocessor macros - macros that define a new struct or structs and functions that manipulate said struct. Hence, to be able to use a container for a specific type, you have to first declare it by calling a macro, like this:

Dynamic arrays

The most common container type, dynamic array (vector in C++ lingo), was also implemented as a macro of this sort. So I decided to go through all of our dynamic arrays defined in this manner and replace them with a stretchy buffer like container, as I recently read Our Machinery was doing.

In case you don't know, a stretcy buffer is a group of C macros you put together to be able to create dynamic arrays on the fly. Instead of declaring specific dynamic array types, you declare a null pointer of the data type your array will contain, and then call macros like 'array_push(some_pointer)' or 'array_count(some_pointer)'. The macros will store the size and capacity of the vector at the beginning of the allocation and assign the pointer to point to the first element. For example:

Stretchy buffers only require a single function, which in our case looks like 'void *darr_ensure_growth_by(void *darr, uint32 grow_by, uint32 item_size)', and a bunch of one-liner macros. That simplifies things nicely, because our previous dynamic array definition macro was over a hundred lines long (it has now been erased), and different array types no longer need to be defined via a specific macro call.

By the way, I also made most dynamic containers have a growth rate of 2, meaning they double in size when they fill up. I thought this to be excessive before, but it turns out many standard libraries do this too. Dynamic strings I gave a growth rate of 1.5. In some places we used to have growth rates as small as 5% which was really bad for small groups of objects, leading to a lot of realloc calls (for example, in our GUI).

Other containers

We have some other container types, too. For example, we have a macro for declaring a dynamic object pool for a type, which allocates multiple objects when it needs to grow and provides a linked list-like interface. And of course, we have hash tables. I didn't do anything to these macros yet, because I wasn't sure if I wanted to, but also because of time. One could build a "typeless" hash table for example, but it would be difficult to choose the correct hash function and bucket size  without declaring them via a macro.

Thinking about dynamic memory allocation

When it comes to dynamic memory allocation, I think I'm somewhere between the extremes of liberal and conservative.  Th C++ way of thinking where everything is allocated dynamically is definitely not for me, and since I only really care about the PC as a platform, I also don't feel like I need to know the amount of memory I need beforehand (although I've done that out of curiosity before). The way I would describe my dynamic memory usage is: if it can be allocated statically, allocate it statically. Do that also when you know the absolute maximum amount of bytes an allocation may require. But it's ok to use dynamic arrays and the like when you really don't know.

That being said, I've been pretty liberal about freeing memory. I always let dynamic arrays and similar containers grow until the end of the program without freeing them in between, because I know at some point they will reach their absolute maximum. But I also have not gotten used to cleaning allocations up at the end of a program, since modern operating systems take care of that for you.

I've taken a step back in this thinking though and decided dynamically allocated recources must still be freed at the end of the program, never mind if the operating system is magically capable of doing that for you or not. That' because in larger programs, tracking memory leaks becomes a bit of a pain if you never clean anything up. Tools that track allocations, like AddressSanitizer and Valgrind, will report all leaks, and if you never free anything, the output won't really tell you much because there's a lot of noise.

So far I have added memory clean up to the MUTA client. Now if there is a leak, at least I will be able to tell where it is. I didn't give the same treatment to the server side programs yet however because the whole server side infrastructure will change drastically as the project goes on.

The case of GUI memory management

A sort of memory-related issue I also dealt with cleaning up was the way our GUI's memory was handled. This was due to an experiment I wanted to make when I first wrote the UI base.

MUTA's UI is an immediate mode style one. In the unlikely case you haven't heard of such a thing, take a look at Casey Muratori's  introduction or the popular Dear ImGui library. In short, instead of creating gui "objects", you call functions like this:

Have you ever though of what would happen if all your GUI memory was stored in a single, contiguous dynamic block? No? I have. I don't know what was going through my mind when I decided to implement MUTA's UI memory management in this fashion, but I was probably thinking of serialization and to a lesser extent, vague ideas of "cache coherence".

You know what happens when a dynamic memory block is resized? Pointers into it are invalidated if the block has to be moved. Storing a complex structure with many pointers like this may not be the best idea. But that's what I did.

What this lead to was that all pointers had to be stored as relative pointers, integer offsets into the main block of UI memory. This block contained everything, including many dynamic arrays. When a dynamic array had to be resized, we looked in the block if there was enough free space - if not, the whole block had to be resized.

It was a real pain, because you couldn't reliably use pointers even inside functions. If you had a pointer to, say, a window structure, and after fetching that pointer called a function that might move the main memory block of the UI (for example, by drawing some vertices), you had to fetch the pointer again using the aforementioned relative pointer. It was painful to me, and it was more painful to other people who tried touching the UI code earlier.

So I did away with all that. Now the UI has normal, individually allocated dynamic arrays. But at the same time, I removed many of the dynamic elements, beacuse they weren't needed. Some elements are now statically allocated, and there's simply a maximum limit to their number. Others, such as vertices and text formatting buffers are still stored dynamically, because it's hard to know the maximum amount of things such as that beforehand, and because I want the UI to be reusable in other projects.

Immediate mode GUIs are nothing new but I kind of want to write a little about MUTA's specific implementation at some point. Maybe I will, though there are still some features that need adding.

I've uploaded the current GUI API and implementation files here and here in case you're interested.

Adding creatures and NPCs

Cleaning up is the bulk of what I have done lately, but not everything. I added some messages concerning creatures to our protocol, which is rather easy using the MUTA Packet Writer application written by Lommi. The packet writer let's us define packets in a text file, from which it produces serialiazation functions and struct definitions.

So far I only added spawning and despawning, and made sure those work with interest areas (structures that define to whom information of nearby events is sent). Plus a GM command for spawning creatures to test things. But the system is surprisingly complicated to implement, because we will need a concept of what spawn points are, how creature IDs are handled, etc. We don't just want to spawn creatures into the world without any context - that may lead to bugs like unique NPCs spawning twice, or something worse. We also need a format to store those spawn points in, and lastly we're going to need scripting. The latter will probably be done with Lua.

Asynchronous asset loading

The asset system has seen some grand internal upgrades. Textures and sounds may now both be loaded asynchornously. This was possible before, but only in a half-assed, buggy kind of way. There's still work to be done here, but it kind of works.  There's also a sort of garbage collection mechanism in place which gets rid of unused assets after a while.

The asset API remains about the same as before. It looks something like this:

Clean up takes time, perfection is impossible

Everyone in software knows this, but there is no such thing as perfect code. There's always something to improve upon. And that's definitely something I came to think of once again while going through our codebase.

There are so many things I'd wish to do better. I want to clean up formatting, I want to touch this and that irrelevant "issue" because I know there's a more optimal solution. For example, I could do away with some slight branching in the updating of moving objects on the client side by separating the objects into two different arrays: idlers and movers. But this kind of thinking is treacherous. It's a 2D game, with only up to maybe a thousand mobile objects on screen at once. It just doesn't matter. But the engineer inside most programmers craves to take on the most minimal of issues.

Beautifying code is as treacherous as over-engineering. I could spend years making the code prettier, but if it does not significantly reduce the amount of time it takes to upkeep said code, it just isn't worth it. So, in the near future, MUTA's codebase will retain it's ugly macros and similar beauty flaws.

When the urge strikes to spend time on something that isn't worth it, I try to think of existing games. I'm certain no successful game has ever shipped without it's programmers thinking they could've done a lot better if only they'd had a little more time. Especially I think of Ultima Online, which was one of the first graphical MMOs and groundbreaking in a lot of ways, and yet most of it, according to Raph Koster, was written by a single programmer over the span of as little as three years. Now that's one heroic feat even from  a larger team of programmers, but it also signals to me that the codebase must have been pretty damn far from perfect.

Note: this post contains personal opinions reflective only of the author's views and not necessarily of anyone else who partakes in the upkeep of this website.

Often times nowadays, politicians in democratic societies appear as if they have completely abandoned ideology. Ideology is often frowned upon, at least if it isn't of the kind that emphasizes the growth of the markets, increasing the amount of foreign investments in the country, or something in that fashion. But emphasizing these points, of course, is not viewed as ideology at all - it is merely viewed as pragmatism through the narrow lens of market centricism.

But sometimes ideology still makes an appearance. You've probably heard of many EU countries deporting Russian diplomats because of the poisoning of Sergei Skripal in Britain, an incident the culprit of, the UK and US claim, is Russia. Finland is amongst those countries.

Now, I don't know who poisoned Skripal. But I believe that neither do Finnish officials. All they have, at least so I believe, is the word of two NATO countries, the US and the UK. And how trustworthy exactly is that word?

In the traditional nationalist Russia-fearing Finnish mindset where everything that comes from the west is superior, held on to by a certain portion of the population, the word of said countries is apparently very trustworthy, at least if it's an excuse to provoke the grand eastern neighbour (which is rather analogous to playing with fire). Never you mind the UK only gave Russia a single day to respond to the accusations they made, or the fact no evidence of Russian interference has been released to the public.

I feel as if Finnish politicians, those who are afraid of Russia at least, are forgetting what the best strategy has always been for "defense" in this small country between two great powers: neutrality. And diplomacy. Not only are we now making decisions based on no evidence, but we're risking our eastern relationship in the name of the ideology of West is Best, and not even for the first time.

Of course, if it is proven Russia was behind the attack, it's a different matter. But this hasn't happened, not so far at least.

Ain't not rewriting it in Rust

Who hasn't heard the phrase "rewrite it in Rust"? I certainly have, and yesterday finally thought, "you know, maybe I should give the language a try - a safer version of C sounds pretty tempting". So I installed rustc, opened a quick tutorial from the web and dived in at the shallow end.

I got as far as structs in about 5 minutes. So off I went and declared my struct:

I'm sorry, but I don't think its the job of the language to tell me how to name my data structures or variables, other than for things like whether they can include spaces/numbers or not, etc. I can go along with indentation marking scope, like in Python or other languages - I believe that's been proven to improve readability across the board. But I'm not interested in styling my naming conventions after the personal tastes of the creator of the language.

I realize styling warnings can be disabled via compiler options, but if its in the standard, you probably shouldn't. So I guess that's it for Rust from my part for now.

Unity's C# source code reference-only release

Unity recently decided to release it's engine and editor C# source code. The C++ core of the engine remains out of sight, and even the C# release is made under a reference-only license, meaning that modifying the source is not allowed. So, unfortunately we don't come even close to fulfilling the four software freedoms as traditionally defined. Unity seems to stress this themselves in the blog post. And of course, one may argue this means little since the C# source code could already be decompiled by parties interested. But I won't deny it's still a very small step in the right direction.

The release, I think, is not that big of a deal though. Unortunately, Unity still appears to hold some old-fashioned views popular in the proprietary software sphere, as proven by this statement, taken directly from the Unity blog: We’d open source all of Unity today if we thought we could get away with it and still be in business tomorrow.

I don't think that statement makes much sense. Of course taking a more open approach would require slight adjustments in the way the company does business, but Unity is already in the market of software as a service. It's not just the engine itself, its the documentation and updates people want. Even if they had a fairly open license that allowed reading, modifying and distributing source code, Unity could keep on raking in the money from the 5% they charge from people who make games built with their engine. The source being open would, I believe, really have no negative effects from this point of view.  Also, knowing Unity's limitations (which are many), I'm fairly sure there's no such advanced technology inside the engine that it would do much for a competitor to read the code, especially considering their main competitor, Unreal Engine 4, already has it's source visible.

Well, here's to hoping software vendors slowly come to their senses. Free/open source is not only ethical, but also practical - knowing your underlying technology from the inside out certainly makes it easier to make a good game.

I believe wrote in an earlier post that I had lately been working on a (proprietary) database server with MariaDB. I have little experience with database programming, so I've had to look a thing or two up on the internet and documentation. What I'll write about here is, I'm sure, very old news to anyone who's done work in this area before, but to me it was fresh knowledge.

So one thing I noticed about the MySQL C API, which MariaDB also uses, as I expect anyone who starts working with it would notice quickly, is that it isn't exactly asynchronous. mysql_query() is a blocking call that will connect to the server, send out the query and then wait for the reply, all in one function.

I didn't want to use the API like this if I didn't have to, and thanks to the internet, finding a solution wasn't too time-consuming. That's thanks to Jan Kneschke who has written about a fix right here about ten years ago.

As it turns out, the MySQL C API has a bunch of less documented functions available through the headers, namely 'mysql_send_query()' and 'mysql_read_query_result()'. With these functions available one can:

   
1. Initialize multiple MYSQL instances (create multiple connections),
   
2. Register each MYSQL struct's socket file descriptor with an event    interface, such as Linux's epoll,
   
3. Send a bunch of queries, each through a different connection using    mysql_send_query(),
   
4. Wait on the event interface until an event happens on one of the file    descriptors, then call mysql_read_result().

The big thing here is opening up multiple MySQL connections and monitoring them all with a select()-like interface. In (more or less pseudo) code on Linux:

Now I thought that was pretty handy. On Windows, I expect you could get similar behaviour using select(). This approach has worked well enough in my current project so far, although obviously things aren't quite as simple as in the example.

The edge-triggered listening socket incident

I'm by no means very experienced in asynchornous programming (or any specific area of programming for that matter). So, writing a server program last week I ran into an interesting bug with a listening socket tracked by an epoll instance, which was really caused by me not thinking of the flags I passed to epoll properly.

The issue was this: a thread was sleeping on an epoll_wait() call until either one of the client's would send something, or a listening socket would receive a new connection that needed to be accept()-ed. When an epoll event on the listening socket fired (a new connection arrived), a notification would be passed to another thread in the program, which would then call the accept() function. An important bit of informatoin is that there was an atomic counter for how many times accept() had to be called. Every time an epoll event was generated by the listening socket, that counter would be incremented, and it was by this counter the other thread would then call "accept()" when notified - a counter of 10 meant you had to call accept 10 times.

But the accept queue on the other thread was being starved - clients constantly had to wait for long periods as the server would simply only accept one client at a time after having been running for about 15 minutes. Obviously there was a problem somewhere.

The solution: don't use the EPOLLET flag when adding the listening socket to the epoll instance.  See, when the EPOLLET (ET for edge-triggered) flag is defined for a file descriptor as it is registered with epoll, events will only fire if there were no previous unhandled events. So if accept() wasn't called immediately after an event and another connection arrived, no new event would be generated for the second new connection. Thus, our counter for how many accepts had to be called would not work

Eventually I added another counter measure, which was to call accept() on the epoll thread immediately after an event and then pass the resulting file descriptor on to the other thread. This way, the listening socket's backlog shouldn't fill up before accept is called on another thread even under high load.

Thank goodness for AddressSanitizer

Async bugs aren't the only bugs I've been fighting with this week, although this one was also obscure due to things having happened on multiple threads. Just two nights ago I tracked a heap corruption bug I thought was either happening because of OpenAL or the code calling it (which I didn't write) for multiple hours of work time.

But compiling with gcc's -fsanitize=address flag, I realized it was just my own old code, in a completely different place. Oops! Good thing there are tools for easily detecting problems such as this nowadays!

Note: this post contains personal opinions reflective only of the author's views and not necessarily of anyone else who partakes in the upkeep of this website.

The attempt at poisoning Sergei Skripal, an ex-spy who used to work for Great Britain, has been on the news lately, and one can't help but wonder at the timing that coincides with the presidential election in Russia.

The UK and the US have quickly concluded Russia was behind the attack, or at least that's what they say they have concluded. But my view is that it may be worth taking these conclusions with a grain of salt. 

It has been stated that the neurotoxin used in the incident was of Russian origin, and that may well be. But does that not sound a little fishy? I'm no expert, but I'm sure there are plenty of simpler and less suspicious ways to get rid of a person, ways that don't directly point towards the culprit.

We all know that the East-West relationship hasn't been exactly at it's peak as of late. In the US, the debate/investigation on "election hackers" still seems to go on, or so I casually observe. Now, maybe it's just me, but what I see here is a potential motivation to stage Russia to have a part in something nasty.

At the same time, Russia does have an election going on, and it does seem to me like it may be a fruitful time to come up with some East-West conflict that may benefit the current poll-favourite. Then again, I doubt he's really going to need the extra credibility to win.

What ever the truth is, conflict between nations isn't nice, and neither is somebody being murdered.

I've been lazy in terms of working on projects outside of work for the past month or two. I have not written proper updates for MUTA in probably over four weeks and before the ongoing week 10, I also had not touched any other non-work related programming projects.

That's not to say I haven't been programming.  Since the beginning of 2018 I have written a backend server for a game-related project (in C with MariaDB), and I have partaken in a collaborative game engine project (in C++). I just haven't touched any projects I have proper personal investment in, that's all.

But that's about to change now. I'm looking forward to getting back to MUTA, for which I will likely begin writing entity scripting next. And this week, I've again been working on Jeesgen, the static website generator I wrote for the upkeep of this site.

New features

I knew a couple of features would be needed to make the site more functional, so I implemented them this week.

Individual post pages

It is now possible to generate individual pages for each post. This allows permalinking and easier linking to articles in general. It also helps RSS generation.

Whether or not individual pages will be generated for each post is controlled by a config option in the jees_config.cfg file ('generate individual post pages = true/false').

Unlisted pages

Pages in this context are individual areas of the website which all share the same template (for example, they all have the same menu for site navigation, but different content). Previously if a new page was added to a Jeesgen project, it would automatically be added to the default navigation menu. After the update, in the original post file it is possible to specify the option 'create menu entry = true/false' to ignore the creation of a navigation menu item.

This feature is handy when you want to create a page for something that doesn't need to be accessible from everywhere. For example, I may want to create a page for MUTA, but not have it appear up top there amongst the "News", "About" and "Projects" buttons.

Upcoming features

I didn't write Jeesgen to be used much by anyone but myself, although I want it to be applicable to a multitude of different types of websites. So some fairly important features are on the hold until I get a hold of myself:

   
• Proper date-time formatting,
   
• Characters outside of the standard ascii set,
   
• Install scripts for GNU/Linux and Windows,
   
• Distributable binary
   
• Daylight saving time

A feature I'm having problems with figuring out how to properly implement are RSS item descriptions. I like to include HTML in my posts so I in order to copy, say, the 256 first characters of a post to the description, I would have to make sure any open HTML tags are properly terminated so that they don't mess up the RSS XML. I don't see an easy way out of this, though one option could be to make the post format support Markdown, which would mean I would not have to use HTML to format posts.

Other stuff: music I've listened to recently

This isn't really related to programming or video games, but I like music, and especially I'm a big listener of electronic music. So in the name of spreading good music, below are a couple of interesting songs I've had the joy of listening to in the past few months.

Jonas Steur - Silent Waves (Youtube)

I don't know if the subgenre is Balearic or something else, but this one's a wonderfully calming track that I don't remember having heard of until recently, and neither have I paid much attention to the artist in question before.  While the calming melody feels ambient, it seems to keep on playing even after the song is over.

Anetha - Disinhibition (Youtube)

A fresh acid track that has an nicely driving bassline, giving the feeling of the beat crawling out of the listener's own head.

Spy 71 & Eva - Take Me ... Mr Love (Youtube)

An italo classic and one of the songs of that era I constantly return.

Sixteen Souls - Late Night Jam (Youtube)

Some slightly funky, chilled house music from 1998 and nothing else.

Sophie Ellis-Bextor - Murder On The Dancefloor (Youtube)

Just pop music from the early 2000s. But with a disco/disco-house twist! And the music video is great.

While programming as a an activity is something I, as many others, derive joy out of, there are obstacles that make it less appealing from time to time. One of those obstacles is C++. I know picking on C++ is a bit of a fashionable thing to do at the moment, but it's no wonder why: the language is easy to pick on (and it's widely used, for what ever historical reason). Well, lately I've been involved in a project utilising sepples and it's features heavily, and here's my latest gripe with the language: headers polluted with implementation details.

In C, if you want to write a static function inside your source file, unknown to the header file, and the function manipulates some data structure defined in the header, you can. For example:

foo.h:
----------------------------
struct my_struct {int x;};
----------------------------

foo.c:
----------------------------
static void
_modify_struct(struct my_struct *s)
    {s->x = 5;}
----------------------------

So yeah, I'm not a fan of sepples.