ScummVM translation files

This week there was slow progress on my task. I was also doing some miscellaneous things that related to my task.

The very first thing I want to say: My RTL-GUI PR has been merged into the master branch of ScummVM! Hooray!

Since the previous work was also related to GUI, I had to rebase my branch to even out with master and merge the RTL-GUI work into my own. There were many merge conflicts, but I was able to solve them all! Now that I think about it, merge conflicts are not as scary as I thought they were before. They are pretty simple!

I've set up Travis-CI to build my branch. I did this because continuous building on my machine was very slow, and with Travis, it is very easy to catch up on errors, correct them, and so on. After rewriting history several times, I was able to solve all errors (~50) that didn't appear on my local machine. I use MSVC, so many instances of errors don't appear on my machine - even though it builds and runs completely fine.

This week, criezy has helped me a lot. One of the things I didn't know how to deal with was setting and getting texts from Clipboard for the MacOSX system. The backends use objective-c for this, and I'm not exactly familiar with it. I tried looking up some documentation, but without using it myself in ScummVM, I can only get so far. creizy left some comments and examples in my macOS commits, and I used them to update the code, and MacOS build on Travis was finally successful!

I also had some new things to do this week. ScummVM has a lot of accessibility, with that, it also has many languages available for users. These translations are stored as .po files in a separate folder. ScummVM has dev-tools for developers, and among them is a create_translations tool which generates a binary file from the given PO files. This binary file is then read in ScummVM to properly place translated strings.

I had to convert these PO files to use UTF-8 encoding, and adjust everywhere else when it is being written or read. After I converted the encoding, I changed the create translation tool to accept UTF-8 charset files and generate a new binary file with all the changes. Next, I modified the part where ScummVM reads this binary data.

I did the whole thing rather slowly I would say because I was often confused. At the end of the day, working on it slowly step by step, I was able to display the languages properly!

However, only some languages worked and the others caused a crash. It seemed like the first few languages written in the binary file worked, but after a specific point, it stopped working.

I honestly couldn't figure out what was going wrong. If the first few languages are working, why did others crash? All the languages wrote equivalent data to the file, so I couldn't make sense of it. creizy helped me out with this and pointed me in the right direction. After a while, I still couldn't figure it out. I asked again for some guidance, and creizy asked some questions, gave some hints and it was really a fun way to follow along. In the end, I got what was wrong!

So, at the beginning of the binary file, the tool writes how long does each language last for. In other words, it helps the reading code to jump to the location where a specific language begins.

For greek, this size was around 95519
However, when it was read, it was 29983.

These block sizes are written as an unsigned 16-bit word. But, the size of greek strings - 95519 - in binary was actually 17bits. This caused the highest bit to be lost and the value thus became 29983 in the binary file. I then wrote these block sizes as 32bit words, and the problem was solved!

That's it from me this week. I will see you next time!

Thanks for reading!