We fulfill power fantasies

RSS Feed

Hello proxy

24.11.2018 16:17:03

This just in: MUTA shards can now only be connected to through the MUTA proxy server (still in the login-branch of the Git repository). Pictured is the first play session through a proxy, the server applications running in the terminals above.

Yep, still the same old unfinished developer graphics though.

Buy nothing day

23.11.2018 07:21:24

It's that time of the year again: the year's biggest celebration of consumerism, Christmas, is looming over. But even here in Finland it's not just Christmas anymore that makes people go and mindlessly buy things they don't need; no, we need to start the wasteful buying of things early before December. The American phenomenon of Black Friday has made it's way over here, machined by the merchants who know well how to exploit the generations that grew up on the internet, learning everything about all the cool American "traditions".

If you're about to take part in the consumption mania of today, I would ask you to consider the following: make it a buy nothing day instead. At the very least, ask yourself the following questions.

  • Do you really need the thing you thought of buying today? Would you buy it even if it wasn't on sale? If you have an item similar to what you're about to buy, still in good condition, do you really need a new one this soon?
  • Consider what you might do with the money saved if you didn't spontaneously spend it on things just because they were on sale for one day (or in the case of Finland, for a whole week!)
  • Think about what the impact of overconsuming things such as electronics is to the environment.

Happy buy nothing day.

Event-based server architecture

22.11.2018 12:10:49

What's the right way to do multithreading? There's not one single answer to this as every program has it's specific needs and purposes. But on a per-program basis, one approach will probably be better than another.

Game developers not working on high-end engines rarely have to worry much about multithreading. Even if a game is multithreaded, at the core most games' logic is single-threaded.

For instance, if your game has a dedicated rendering thread, your previous frame might be getting rendered while you simulate the current one on the main thread. But what you're doing is you're just replicating the relevant data from the main thread for use by the render thread, thus not affecting the actual game logic, which remains single-threaded.

For another example, your physics system might parallelize a for-loop to multiple threads, all of which are synchronized at the end of the loop. This also does not change your game's logic from single- to multithreaded, you're just iterating some loops faster by parallelizing them.

There's a reason the above two examples are common in games, and it is that they are easy to program for, while getting some of the benefits of dividing work to multiple threads. When all of the real logic happens on a single thread of execution, the control flow is predictable and easy to understand. It might also be more efficient than a truly multithreaded alternative, in case the multithreading is done badly, or the algorithm is not suitable for threading.

Networking and threads

While you can well write a game using just a single thread, writing network code without threads is more difficult. This is because in network programming many blocking calls are used, such as poll() or recvfrom(). Usually, we want a thread to sit there and wait until network messages or new connections arrive. But a sleeping thread cannot do much else, so the other logic must happen on another thread.

It is usually possible to give blocking functions a timeout, so that even if they have no new events to report to us, they will return after a given amount of time. In this way, one can imagine a game-loop-like server architecture that polls for messages and new connections, all the while also executing all of the logic on the same thread as well. With a timed wait, we could have the loop run at approximately, say, 60 FPS.

But servers that want scalability probably don't run on a single thread. I'm no expert of course, so take this with a grain of salt, but so my investigations on the internet suggest. It seems common to have threads dedicated simply to polling for new network events.

State makes things difficult

If the only job of one of our threads, the network thread, is to sleep on poll(), epoll() or similar call and wait for new incoming messages and connections, the logic of the program is handled somewhere else. Instead of interpreting incoming data right there and then on the network thread as it arrives, the interpretation happens on a different thread.

Applications such as game servers use long-lasting connections instead of quickly terminated ones. For long connections, the server has to store some kind of per-client state: the socket of the client, a session id, incomplete messages, game character variables, etc. After acknowledging we have a lot of state, the question becomes: where's the right place to manipulate it, considering our program is multithreaded?

To store the state of our clients we might have a struct called client_t. We also have some kind of a container that contains those structs. Lets say the structs are stored in a pool of a fixed size.

typedef struct client_t
{
   int     socket;
   uint32  session_id;
   uint8   unread_messages[512];
   int     num_bytes_unread;
} client_t;

client_pool_t clients;

/* Returns NULL if no free clients are left in the pool */
client_t *new_client(client_pool_t *pool);
void free_client(client_pool_t *pool, client_t *client);

With the above premise, let's assume our network thread accepts clients and reserves client_t structs for them. If the clients disconnect (recv() returns 0), the network thread frees the client struct.

Now, if all of our connections need to be stored in our global client_pool_t data structure, how do we access it safely from multiple threads? Remember that the network thread does not handle any of the actual (game) logic in the program: that happens elsewhere, and that logic also touches the client structures as it must send messages to the clients or interpret their messages stored in the unread_messages buffer, etc.

The naive answer to safe access is to use a mutex. Wrap the new_client() and free_client() calls to a mutex and you're fine. Now when the network thread accepts a new client, it can safely call new_client() and then somehow signal the main thread that a new client was accepted, passing a pointer to the newly allocated struct along.

There are two problems with this approach: the possible performance concerns, and the (in my opinion) more important topic of code complexity. These problems are somewhat intertwined.

What kind of problems might the above approach lead to? Say that at some point in development you realize that there are situations where, on the main thread, you have to iterate through all of your clients. For that, you might create an array of pointers to the currently used client structs. So now, when you accept a new client and reserve a new client struct from the pool, you have to also push a pointer to the struct to said array.

But now, what might happen is that while the main thread iterates through all of the clients, one of the clients disconnects. In this case the networking thread frees the client struct of the disconnected client. Since we don't want this to happen while we're iterating through the clients, we have to lock the pool's mutex in both cases: while iterating, and while deleting a client.

The possible performance implications of the above are easy understand: the iteration can leave the mutex locked for a while, meaning the networking thread gets blocked until the iteration finishes.

But more importantly, using the above approach we are forced to constantly remind our selves of the fact we can't just go doing anything to our clients on the main thread without remembering the mutex and the possible race conditions. As we add features, the code becomes more and more complicated, and easier to make a mistake at. The above was a simple and somewhat dumb example. For another example, consider what should happen if the networking thread writes recvd() data into a client's unread_messages buffer, and the main thread handles that data. Are you going to use a per-client mutex to protect access to the data buffer while it's getting written to or read from? That's gonna be some more complex code using locks.

To some extent I'm speaking from experience here. When I started writing the MUTA server, I had little clue about how to write a complex server capable of handling a high amount of clients. I'm not sure if I still have the right idea of that, but I know that multithreading was a pain in my ass for a long while, and to some extent it remains so, although I think it's gettin getter.

Handling state on a single thread

So far I've found that the simplest way of dealing with the above woes is to be simple and handle all state changes on on a single thread. No allocations of stateful data structures or otherwise manipulating them from the networking thread or anywhere else. I've referred to this as event-based architecture, but I'm not sure if that's the correct term; all I know is that I've heard the term elsewhere and assumed this was what they meant. Then again, who cares what it's called.

In this model, when a network thread receives a message, it will not handle the message immediately. Instead it will pass it on to another thread as an event. The thread that handles the events (usually the main thread) can sleep on a blocking function from which it wakes when new events are posted from another thread.

Essentially, both the main and network thread will sleep in this model until something relevant to them happens. Of course, you can have more than one thread pushing events to the main thread though, and you can also have the main thread posting events to other threads. MUTA's login server has a database thread with which the main thread communicates in a two-way fashion, posting and receiving events from it.

Current implementation in MUTA

MUTA's server consists of multiple programs now: the login server, the shard server, the world simulation server, the database server and the proxy server. The two newest applications, the login and proxy servers, implement the architecture described in this post.

The sleeping and waking of a thread is implemented using a condition variable that sleeps on a mutex. The same mutex is locked when items are pushed to the event queue.

The event queue is also a blocking queue. It has a fixed size, and if it fills up, meaning the thread pushing events is pushing them faster than the main thread can handle them, the push function will sleep until space becomes free. This is also implemented using a condition variable.

The event data type of the MUTA login server looks as follow at the moment.

enum event_type
{
   EVENT_ACCEPT_CLIENT,
   EVENT_READ_CLIENT,
   EVENT_ACCOUNT_LOGIN_QUERY_FINISHED,
   EVENT_ACCEPT_SHARD,
   EVENT_READ_SHARD,
   EVENT_LOGIN_REQUEST_RESULT
};

... various event structures ...

struct event_t
{
   union
   {
       int                                     type;
       accept_client_event_t                   accept_client;
       read_client_event_t                     read_client;
       account_login_query_finished_event_t    account_login_query_finished;
       accept_shard_event_t                    accept_shard;
       read_shard_event_t                      read_shard;
   };
};

As an example event, we can take the accept_client event, which is fired when the network thread accept()s a new connection. It looks as follows.

struct accept_client_event_t
{
   int         type;
   socket_t    socket;
   addr_t      address;
};

As we can see, the event data type is a union. That's because they all get posted into the same array. But what about events that have to transfer variable amounts of data from thread to thread, such as socket recvs() (that is, packets received from the network)? In such cases, we use a simply mutex-locked allocation pool. Lock the allocator's mutex, allocate data, unlock the mutex. At read, we free the allocated memory back to the allocator in a similar fashion.

The allocator used here is a simple segregate fit -type of allocator. The logic is this: on malloc-requests, round up the requested amount to the nearest power of two. Find out the highest (leftmost) bit of the amount and use it as an index into an internal pool of memory blocks of the given size.

While the allocator calls real malloc when it needs more memory, it has an upper capacity to which it will allocate, since the event queue is blocking. Thus it is not possible to DDOS the server to run out of memory by simply spamming packets at it.

The event API itself looks as follows.

typedef struct event_buf_t event_buf_t;

void event_init(event_buf_t *buf, uint32 item_size, int32 max);
void event_push(event_buf_t *buf, void *evs, int32 num);
int event_wait(event_buf_t *buf, void *evs, int32 max, int timeout_ms);
/* -1 as timeout means infinite */

And, using the API, the login server's main loop looks as follows now.

for (;;)
{
   event_t events[MAX_EVENTS];
   int num_events = event_wait(event_buf, _events, MAX_EVENTS, 500);
   for (int i = 0; i < num_events; ++i)
   {
       event_t *e = &_events[i];
       switch (e->type)
       {
       case EVENT_ACCEPT_CLIENT:
           cl_accept(&e->accept_client);
           break;
       case EVENT_READ_CLIENT:
           cl_read(&e->read_client);
           break;
       case EVENT_ACCOUNT_LOGIN_QUERY_FINISHED:
           cl_finish_login_attempt(&e->account_login_query_finished);
           break;
       case EVENT_ACCEPT_SHARD:
           shards_accept(&e->accept_shard);
           break;
       case EVENT_READ_SHARD:
           shards_read(&e->read_shard);
           break;
       default:
           muta_assert(0);
       }
   }
   cl_check_timeouts();
   shards_flush(); // Flush queued messages to shards
   cl_flush();     // Flush queued messages to clients
}

And so, most logic is handled on a single thread.

Converting other systems to the same approach

The MUTA server's older parts are not implemented using the model discussed in this post. Although they also handle most logic on a single thread based on events generated by other threads, the model they use is not quite so refined. Instead it is kind of an ad-hoc implementation of the same idea, so instead of having a single event buffer, there are many. These buffers are not blocking like in the case of the login server how ever.

The old parts of the server are also frame-based, in that their main thread runs the logic in specified increments of time (FPS, frames per second). With an event queue the thread can sleep on we can remove some of the latency provided by the frame-based approach. To make timers based on delta-time still function correctly, we can simply calculate a sleep-time for the event_wait() function much like we would sleep during a frame if we were waiting for vsync in 3D program.

I'll be looking at converting the old programs to this model at some point in the distant future, unless I come across something better.

Give me proper networking tools on Windows

10.11.2018 21:44:09

Last week I spent multiple days creating a bad, inefficient abstraction over WSAPoll, which itself is a bad, inefficient abstraction of other functions, simply provided for UNIX compatibility by Microsoft.

The thing is, Windows has efficient, scalable methods of managing network IO, namely, IO completion ports. But it's such a pain to use in a cross-platform application. With IOCP, because you only get informed after a recv() on a socket is done, not when you can do one yourself, you must maintain unique buffers on each one of your sockets. And if your application is a cross-platform one, you basically have to make your Linux abstraction work in a similar fashion, because the socket recv() function just cannot be used the same way with both systems. (As a side note, I've built an API for async socket IO that works on both systems, using IOCP on Windows and epoll on Linux. It's called netqueue and it's included in MUTA's source code.

Linux on the other has epoll, one of the comfiest APIs I've every used. I love epoll in terms of how easy to use it is. Add a file descriptor or many to an epoll instance and just sleep on epoll_wait() until something happens, like you receive some data over the network.

I want epoll on Windows! But I can't get it, not without additional dependencies to external libraries or a lot of work.

My saving grace is that our game's servers don't need to run on Windows in release mode. The only reason the server runs on Windows is that sometimes we have to develop on it, and in such cases its handy to have the server run on the same machine with the client. So what I've done is I've wrapped WSAPoll to look a bit like epoll. It's a terrible hack and certainly inefficient, but it'll do for our purposes. Of course, features such as EPOLLET, edge-triggered behaviour, will not work on Windows (in edge-triggered mode, new events for sockets are only notified of once, not until the socket is recv()d from), and in fact even a library I found, attempting to provide epoll for Windows, didn't support said feature.

Still. I want epoll or something similar to it on Windows. Why does Win32 programming have to always be such a pain?

MUTA devlog 4: proxy and login servers and future art direction

2.11.2018 17:48:17

The past few months have granted me little time to work on MUTA, our 2D MMORPG. There have been events to attend (NGS), another trip to India (IGS), and work. But I've done a bit of MMO work nevertheless.

The proxy server

I began working on MUTA's proxy server. It is a server that sits between the client and the game server, dispatching messages in both directions. The purpose of it is to never reveal the true IP address of the game server, making it more resilient towards DDOS attacks. Each proxy could be connected to multiple game servers, and each game server to multiple proxies, so that two people playing on the same server might be connected to completely different IP addresses.

I'm writing the initial implementation using TCP, in C as the rest of the game is written in. Adding another layer of connectivity, especially with TCP, will of course increase latency, so it remains to be seen if we should do the game-server-to-proxy connection via UDP later.

The idea for the proxy came from many sources, but one of them is CipSoft's Matthias Rudy's GDC talk about the infrastrucuture of Tibia. The slides for the talk are here.

The login server

Once I had sat down and begun writing the proxy server, I went as far as accepting clients and reading their packets before I realized I should probably write the login server at this point, too. Currently in MUTA, the game server acts as a temporary login server. It wouldn't make much sense to write login-related functionality code between the proxy and game servers, only to remove it later. So I also started up the login server project.

Event based architecture

On the login server I thought I might try out something that's new for me. I'm not a very experienced network programmer, and hence I haven't had the chance to try out that many different tricks yet.

Dividing work between threads is still an issue I constantly run into when writing server software. Usually, because of my background in game programming, I've simply settled for having a main loop that runs at a certain frame rate, and the other threads feed work to that thread, such as handling incoming accepts or reads. Work is fed using various buffers that store different kinds of events (for example, a client received something in it's buffer, the message should be handled). This is not very elegant or energy-efficent design.

On the login server, I thought I might still have one thread executing the main logic, but instead of locking the thread's frame rate, I'll try having the main thread sleep until it gets work from other threads. I will also try to unify the event buffers so that most events would be inserted into a single queue. Insertion would block the feeding thread if the queue is full. Well, we'll see how that goes.

Art direction

Something thing we've recently (and before, too) been pondering with Lommi is what would be the most practical direction we could take in regards to art in MUTA. It's been a tough subject!

We know the feeling we want the game to go for: the sort of gritty fantasy provided by Robert E. Howard's Conan stories and other sword-and-sorcery works. We also really dig the artstyle of many of the comics made of Conan. In a 2D game that would look awesome. But 2D art in 8 isometric directions is painstaking, and it's complicated by things such as displaying equipment on characters while keeping animations smooth-looking.

We thought of a couple of directions we could follow. One would be to reduce the game's resolution from 64 pixel tiles to 32 or 16 pixels. Such a low resolution would allow even us programmers to make art for the game, albeit low quality art. We didn't really like the idea, as it doesn't let us present the game world as such a gritty, "mature" type of place as we'd like, and it would limit the art style a lot. Besides, the programmers probably shouldn't do the art anyway.

Another thought we had was going 3D. Neither of us two working on the game can do 3D art, but 3D would mean not needing to draw 8 different directions for each object (or at least each character). Then again, 3D would probably present our style in quite a bit of a worse manner than well-made 2D in isometric space.

We didn't decide anything yet, other than that the direction we have sought so far would look the best.

Going to India Game Summit 2018

21.9.2018 21:46:40

As I wrote last month, I travelled to Andhra Pradesh, India during the summer to teach game programming on a summer course over at a university there. At the end of this September (less than a week from now) I have the pleasure of going back to the country, this time as an organizer for the first ever India Game Summit.

The event this year will be a game jam, held at four different universities and colleges across the country simultaneously. How ever, it is in the hopes of organizers (the main one of whom is Kajaani University of Applied Sciences) that the event will grow into a games industry conference in the future, something akin to the Northern Game Summit we have in Finland, with speakers from different companies and organizations sharing their knowledge.

An interesting point about organizing a game jam event in India is that it seems to be a fairly foreign concept, at least at universities (including the capital area). Even the term game jam has been mostly unheard of; hence, we've dubbed the event a hackathon instead - a term familiar to most IT people. Organizers have also been contacted by to-be participants worried if it is even possible to build a game in 2 days. It may seem funny to a regular game jammer, but then again there are monetary prizes on the line here.

The event will begin on 28th September and end on the 30th of the same month. I myself will be travelling to Aditya Engineering College in Andhra Pradesh. I'm quite excited and interested to see how things work out, to be honest.

New website logo

In other news, this website has a fancy new logo. Our friend Laura made it. Big thanks to her!

A Game About Carrots

5.8.2018 16:02:35

I've had little time for hobby programming in the past two months due to working abroad. However, while in India my colleague, Laura (who was also there) told me she wanted to make a game. She began drawing some pixel art and on the off hours, I wrote some good old C code.

The game we began writing is Porkkana, Finnish for the word carrot. The game concept is very small and, uh, sort of unclear. Actually, I'm not sure we completely even know what the game is going to be about yet, other than that you grow carrots in it. And that you feed the carrots to bunnies. There's probably going to be some sort of trouble coming your way though, like carrots dying or getting stolen. Who knows.

Most of the engine code I was able to steal from MUTA, my primary project at this time. I copy pasted the OpenGL rendering and the immediate mode GUI system as well as various utilities I always use from different projects. The gameplay code itself is in one file at this point and less than 1000 lines long. The amount of hours spent on programming the game is actually very low, but every now and then I might spend an hour or two on the code in the evening. The most time-consuming thing so far has been getting Android building to work; it does now, but getting to that point took probably at least a full 8 hour work day's worth of time. If there's one thing I hate in this world, it's the Android native code toolchain that seems to change every year.

I'm also returning back to working on MUTA. Currently I'm working on the client hotkey system and UI Lua API. But there's no hurry here, I work on the game when I feel like it.

Oh, and here are some musical picks that I've been playing too much this summer (mostly house and techno, old and new).
Kangding Ray - Summerend
Paul Johnson - Music's In Me
DJ Metatron - U'll Be The King of The Stars
Gianluca Nasci - Stimulation (Stanny Abram Remix)
System F - Dance Valley Theme 2001

Taught Game Development in India for 2 months

1.8.2018 17:11:59

A couple of months ago a friend of mine informed me that my school, Kajaani University of Applied Sciences (KAMK), was going send a delegation to the state of Andhra Pradesh, India, to teach game development for two months. The event would be known as IGDC, short for Indian Game Development Challenge. KAMK was recruiting for the job so I applied and was accepted. I left Finland at the beginning of June and came back home just a day ago, 30th July. I will now attempt to describe the experience in words.

Note that this text is written from my personal perspective and does not represent the views of KAMK nor anyone else who isn't me.

Preparing and leaving for the trip

At some point in time KAMK, my school, had come in contact with APSSDC, the Andhra Pradesh State Skill Development Corporation. They had agreed on KAMK providing the programme for a two month summer course on video game development. Initially the idea had been about bringing Indian students to Finland for the summer, but this was soon diverted so that instead of sending Indians to Finland, KAMK would be sending Finns to India as teachers and organizers. A friend of mine asked me to apply to be part of the delegation going to India and I was accepted. The job was to teach and coach students at the SRM University, Amaravathi.

I don't remember when exactly I got to know I was going to India, but at most I believe it was two months or so before the actual date of leaving. I was working 8 hours a day on an engineering project at that point, so I had little time left for preparing course material or the like. The most work I got to do regarding the India project was when me and my colleague were asked to give an online presentation to some of the Indian students along with a questions and answers session. We staid at work until 9 PM making a PowerPoint because we were told the presentation would need to happen the next day at 7:30 AM. I have to say at this point that little to no time for preparation was sort of a defining feature of this trip. Anyway, the Q&A session left me with one idea on the top of the pile: I would have to work on my understanding of Indian English accents. However, it was also nice to get to know how excited some of the students already seemed.

During the same week we did the Q&A session, three people from India came to visit us in Kajaani, Finland. Two of them were from the APSSDC and one represented SRM University, the school where the course would be held. The Indians were taken hiking as well as to the sauna. Later I heard they were also taken to to see a Finnish dump pit (amongst various other things) which I thought was funny as hell. This was my first contact with Indian people, but also the first time I got to talk to our Finnish project coordinator, an all-around nice guy who would play a very important role during our stay in AP.

Finally at the beginning of June 2018 it was time to leave. I left with one other person only instead of the whole team due to various reasons I will not go into now. Our coordinator, mentioned in the above paragraph, was already waiting for us at SRM University Amaravathi, the campus we would spend the next two months at.

Arrival

We arrived near the weekend and soon met with our own man on the spot, as well as the faculty of the school. We got to see the equipment, which seemed satisfactory, but more computers would be needed - we were going to have nearly 500 students (the APSSDC took care of this well later). The lack of equipment at this point (I assume) had to do with the fact the whole school was still being built. In fact, being built is what was happening to the whole city of Amaravathi 24/7 around the university. The story goes, a couple of years ago the state had broken up into two states and the old capital, Hyderabad, staid in the other state (Telangana), where as Andhra Pradesh was going to build it's own capital from scratch. Indeed, every hour of the day the construction of the school and the city were on-going (which did not help our night sleep at all, by the way).

For our first Sunday (the working week is 6 days in India) one of our local contacts, did something very nice and took us to see his home as well as some other places nearby. That was a gesture we very much appreciated as we got a glimpse of what the country looked like outside of the borders of the campus. The area we were staying at was quite rural, so traveling on the muddy roads took a lot of time. We went to a beach as well and were probably the only people there with nothing but boxers on - I learned later Indians swim in full clothing. Of course we also went to a small temple and ate well.

The first week went on fast. I taught the students some basic concepts of game programming, like what is a program main loop, what tools can you use, etc. as well as practical game programming in C with SDL2. We also talked about version control using Git. A lot of information had to be packed into a very short period of time, and I had not had the time to really prepare any materials at home. I still regret the lack of prepared material, but many people seemed to already catch on and start working on their games. I spent a lot of time in the evenings later preparing course materials for the future since I had it fresh on my mind what was going to be needed.

A week later the rest of our team of 8 Finns arrived. Something that made the first weeks rough for us all was that students did not arrive in one bunch but at the rate of about 40 per day. The total amount at the end was about 480. This meant many things had to be taught again and again, often separately to only a few students at a time. Luckily Indians are (from my limited experience at least) social people and often the students would by themselves inform any new arrivals about various important things.

Course structure

Why would a group of Finnish people be sent to India to teach game development? My understanding is that our job was to introduce the students (and the faculty) to a more practical way of learning, since Indian education is still quite conservative, often (again, to my understanding at least) being about listening to the teacher babble while taking notes. Well, our course was certainly all about practice and not about listening to the teacher babble.

We did some lecturing at the beginning in the form if intensive courses, but just like KAMK game dev summer courses in Finland, most of the time the students worked on their game projects in teams of 4-10 using the tools they wanted. Us teachers, we were simply there available for any questions (or queries, as Indians say) they came up with, occasionally roaming around and checking out the progress being made. People caught onto this model surprisingly quickly, although I have some doubts that there were people even at the end of the course who would have preferred a more theoretical approach - different styles suit different people.

Cultural differences

For someone like me who's never been outside of Europe, seeing even a limited part of India is an eye-opening experience. Just like the EU, India consists of various states with different cultures and languages, but being an ignoramus I had little prior knowledge about how the nation was actually organized. Now I know a little more, but just a little - we only staid in one state (AP), mostly at a university campus, apart from a night or two in Delhi (and that in the near vicinity of the airport - the furthest we got out of the hotel was the liquor store).

Still regarding culture, it warms my heart to see a nation not yet completely taken over by American pop culture (Finland is just about a lost cause at this). Of course many (especially men) wear western clothing and some people know the big Hollywood or pop EDM hits, etc. But at least in the rural areas it seems that, despite the massive presence of American companies such as Coca Cola, Pepsi and the like, people still have their own style of clothing, music, films and customs. That is refreshing and gives some hope for the future of this planet to me.

To just shortly list some differences between Finnish and Indian culture: friendliness (Finnish people don't smile at strangers that much); respect (a nobody like me will get called sir or madam in India, especially wile wearing a blazer); respect for other's time (Indians have little to no sense of punctuality); coffee (Indians have to forcefully put milk and sugar into it). I also cannot help but quote a colleague of mine: how do Indians ever get anywhere by walking so slow. And yet, we all share the same basic needs and understanding of human ethics.

Improvements for the future

This summer course was the first in what is hoped to be many in the near future. Because of that our team had to learn many things in practice. Here are some of those things.

Don't make the course a competition

Since the beginning it was planned that we would choose one team from the course who would be sent to the Slush business event in Helsinki, Finland to pitch their project and company. Immediately upon hearing about this I felt it was a bad idea as it would pitch students against one another. And I feel like that exactly happened, especially towards the end. Games are a business but at least in Finland companies tend to collaborate and share information - ruthless competition isn't that big a part of the picture, or at least that's the feeling I get. Thinking of your co-learner or colleague as a competitor only creates bad blood and burns bridges.

Don't overpromise

We tried hard not to overpromise, but sometimes you should just keep your mouth completely shut instead of telling someone of the possibility of something happening. For example, we were expecting three companies from Kajaani to visit the university, but we should not have spoken about it before the companies confirmed that they were indeed coming. In the end, only one of the companies paid a visit (my greatest thanks go to Virtual Frontiers for this by the way).

Let all students show off their progress

We had nearly 500 people working on various game projects for two full months. That's a lot of working hours, especially considering the working week is 6 days in India and the working day seems to be of a varying length upwards from 8 hours. A massive effort from some very dedicated people to say the least.

All project courses at KAMK end in a post mortem session where students get to dissect everything that went right or wrong during the project period. This way students get to learn from others' mistakes and successes, but at the same time, a post mortem acts as a public revelation of a student project to the public - finally you get to show off the hard work you've done, and usually you also get some useful commentary! The post mortem event is one of the highlights of a course.

Because of scheduling problems, our own mistakes in management, travel timings, stress and various other excuses, we forgot about the post mortems at this event. That was a mistake that I feel made many students feel disappointed, to feel as if their work was not appreciated. That still bothers me. When you've made something, you want to tell the world, and you should be given the chance to do that. And of course, we would have wanted to see those games presented, too.

Keep on pitching

When a project course begins at KAMK, students must first pitch their projects to teachers and get them accepted. We did the same thing at the IGDC, but also kept on holding practice pitching sessions every Friday for the full duration of the course. That was great and should be continued - it was really motivating to see the improvements in the presentations made by the students. Some definite performers there.

Regards

All in all, for me the overwhelming feeling left of this trip is positivity. My heartfelt thanks go to the motivated and friendly students, to the helpful faculty of the organizations involved as well as to the various other great people I (we) met on the way.

The greatest motivator for me was always the progress and enthusiasm of the students who I would thank all personally did I have the time. Here's to hoping for more collaboration between Indian and Finnish educational agencies, and for more events of this kind in the future.

MUTA devlog 3: Lua scripting, game data files and packetwriter improvements

12.5.2018 11:18:45

My most recent workings on MUTA have been about adding ways to implement game content. I have now added the first version of the server side scripting language, Lua. In relation to this work, I've had to implement multiple data file formats so that we know thing slike which scripts to load and which game object uses which script.

File formats for defining game objects

The first thing I wanted to try scripting with were creatures. But before I could script creatures, we had to be able to define them. For that a file format was required. I had used ini-type files so far for quick debug purposes, but I decided it was time to implement a better format.

I've written earlier about the binary mdb file format I wrote for use in the game - we use it for art assets. How ever, while debugging and trying out new stuff, binary files with special editors (mdb has a command line editor in which you give commands like "set column") are a little cumbersome. Text files on the other hand are easy to modify on the go. So a text-based format seemed logical for something like creature definitions that - entries that would change frequently.

Definitions like these aren't complex to represent. An ini file would not do it, but something very close to it would. So here's how our creature definition files look like now:

creature: matlock_creature_1
   name            = Matlock
   description     = Tiesin heti et tollanen haluun ison olla
   sex             = 0.5
   attackable      = 0
   texture_name    =
   script          = test

creature: human_male_1
   name            = Man
   description     = A man
   sex             = 0
   attackable      = 0
   script          = test
   #ae_set is an entity animation set (this line is a comment)
   ae_set          = human_male_1

A similar format is handy for a lot of things: creature, dynamic object, static object and player race definitions to name some. So there's a generalized parser function:

int
parse_def_file(const char *fp,
   int (*on_def)(void *ctx, const char *def, const char *val),
   int (*on_opt)(void *ctx, const char *opt, const char *val),
   void *ctx);

As the function parses the file given in parameter fp, the on_def callback is called when a new definition beginning is found, for example "creature: human_male_1". The callback on_opt is called when an ini-like option is encountered. Ctx is an optional pointer to a caller-defined context. The option name and the value are passed as strings to the callbacks. A missing feature for now are multi-line values, which we might need at some point for longer pieces of text like object descriptions. It's not fancy (actually it's almost the same as .desktop files on Linux or many other similar formats), but these files are both, easy to change on the go and human readable.

Lua scripting creatures

The addition of a scripting language is something I've been meaning to get to for a longer while, but everytime I've meant to get to it, I've found some other feature missing and wandered off. A couple of weeks back I finally got my hands dirty with Lua.

I had not used Lua earlier so it took a little bit of reading the reference manual and the Programming in Lua text found online to figure out how to use the thing. Lua is of course one of, if not
the
most popular embedded scripting language, so the process was fairly simple - if it wasn't, I doubt the language would have such a following.

The first thing I decided to apply scripting for were creatures. Here's how a creature script might look like at the moment:

local v = 0

function Init(creature)
   v = 0
end

function Tick(creature, delta)
   --creature_walk walks a creature in the given direction (0-7)
   creature_walk(creature, v);
   v = (v + 1) % 8
end

The Init() function is called when the script it attached to an entity. The Tick function is called every frame if it exists. As more use cases come up, more functions will probably be added. All of the functions will probably be optional, too, so if an entity doesn't require a script function call every frame, the Tick function for example could be omitted.

The MUTA Lua-side API will probably consist of mostly C functions bound to Lua. In the example above for example, creature_walk is a C function binding. Functions like these operate directly on raw entity pointers (Lua's 'user data' type). I don't know yet if that's a good idea, but it's certainly easy. One thing that must be kept in mind is that since our world will be divided into multiple 'worldd' processes, each simulating a part of the map, the scripts must be movable from one node to another and back again.

Defining scripts

All scripts to be loaded at server start-up must be defined in an ini file. Each line of the file defines the name and the file path of a script, for instance, "test = muta-data/server/scripts/test.lua". This associates an easy-to-remember string id with the script. The given name can then be used in game object definition files to refer to the script.

Script hot reloading

The nice thing about scripts is that you can modify them as the program is running. The way this functionality works in MUTA is that a character with the game master status calls a chat command with the script's name.

When a GM types in, '.reload_script test', a command is sent to every worldd process connected to the master server to reload the script 'test'.  Once the worldd's receive the command, they will look up the script, reload it's Lua code and re-register it's functions in Lua's own registry, then find every entity in the world using the script and reload it for them. That's quite nice for fast iteration!

Packetwriter improvements

Elsewhere, Lommi has been working on improving MUTA Packetwriter, the program we use to define and generate code for network packets. The program works so that every packet in the protocol is defined in a .mp file, which is passed as a parameter to the command-line packetwriter program. The program reads the file and forms the correct structs and inline functions for handling the defined packets, writing them into a C header file.  While the changes aren't upstream yet, there's a bunch of nice stuff incoming.

The improvements include packet definition formatting that isn't as strict as it used to be. It used to be that a white space or linebreak in the wrong place could break the generation but the program would not tell you want went wrong, instead just generating broken code.

You're also now allowed to do arithmetic when defining packet specific size limits.  This is a handy feature for packets that contain variable size elements such as strings.

Support for arrays of packed structs is also on it's way. This is a feature I've been wanting a lot because it's damn handy. For example when sending an account's character list, instead of being able to send an array of structs I've had to create an individual array for every parameter of every character. This has resulted in a lot of friction because populating multiple arrays is obviously more complex than populating a single array, and since our way of using packets is fairly low level, there have been bugs with uninitialized variables and such that took a while to debug.

Up next

I'm currently working on improving creatures. I'm not quite sure what exactly I'll be doing next, but it will be about making gameplay implementation easier, be it more Lua API functionality or actual game object interaction between the client and the server. Better get coding right now!

Ultima Online 20 year post mortem online from GDC 2018

3.4.2018 21:05:14

This is really just a heads up for anyone who missed it: this year's GDC featured a new post mortem session for the now 20 year old Ultima Online, and the video is already available for free here.

UO remains a source of interesting MMO history and design topics to this day. Although the session featured many of the same talking points as the post mortem in 2012, there was a little bit of new stuff there, too. For example, this time Richard Garriott himself took part, and Rich Vogel talked about the invention of game time cards.

I only wish I could find an interview with Rick Delashmit, the original main programmer of the game. It would be interesting to know for example how they stored world state information and account data on disk back in the day!

Page: 0 1