Retrospective: Hellsinker Translation Development

So I volunteered to work on a translation for Hellsinker, a game by Ruminant’s Whimper.

“It’s just a shmup, how difficult could hacking it be?”

Famous last words.

It finally got released, and you can get v1.1 over here. It fixes a few minor issues over the initial release. Check it out. Huge huge huge thanks to Nazerine and Halbarad for doing the translation and image editing respectively.

Warning: Spoilers and technical babble ahead! Do not read if you don’t want them.

Also, brace yourself. This is going to be a long one.

Resource identification

Hellsinker is a strange little game, with a lot of mysterious content in it that all had to be tracked down. I’m not going to lie, a huge amount of time was spent in simply finding all necessary resources and replicating their existence within the game itself.

Here’s a basic list of everything we needed to find and get translated or edited:

  • Startup configuration menu.
  • All menu item descriptions.
  • All TUNING DIPSW screen option descriptions and settings.
  • All in-game dialogue:
    DEADLIAR (all segments except segment 1 behind and segment 2 behind)
    MINOGAME (segment 5)
    KAGURA (segment 5)
  • All novel/story dialogue from the strategy recorder:
    20 text/*.CTX files
    (ARSENAL.CTX and int_w_ex.CTX are unused in the game)
  • Segment 5 first run only dialogue.
  • Segment 7 Rex dialogue.
  • Segment 8 bootup procedure.
  • Segment 8 boss pattern typo fix.
  • Shrine of Farewell replay-only gibberish. (It’s completely unreadable…)
  • Shrine of Farewell final boss pattern names. (Also completely unreadable.)
  • All 7 endings.
  • Fix typos in some boss patterns and names.

That’s a lot of little things. You’ll soon see it’s even more than it sounds like.

Less than half an hour after I finished the hacking and reinsertion for all of the above, the original developer decided to release patch 1.0.0.9, after four years of inactivity.

That was the single most soul crushing moment of my life.

While the changelog for it was fairly short, a lot changed under the hood which mean that not only would I be porting everything to the new version, I would also have to rewrite some of what I had already done.

The executable’s resources

Hellsinker is written in Delphi. That’s right, Delphi. I have not seen a Pascal derivative language out in the wild for almost a decade, so this really took me by surprise.

On a random note, it uses a graphics library called Quadruple D for all of its Direct3D integration. Between 1.0.0.8f and 1.0.0.9 he changed library versions to support DirectX 9.

Delphi’s text storage is interesting. Unlike most traditional programs, it stores text data in the same memory segment as code. As we already had insertion tools in place to add a new text segment with our translated text, this had the nice side effect of meaning that any strings we took out of there would mean more room in the code segment for any code we needed to add. Very very helpful.

The odd part here is that the code segment is set writable! Most programs make that memory block read-only because, well, it’s code. It shouldn’t be changing. So why is it writable? Because of the text data. Strings are of the following format:

(addr - 8) : 4 byte lock counter (default: 0xffffffff)
(addr - 4) : 4 byte string length value
(addr + 0) : string text

The lock counter in particular is interesting. Any time the code needs to access a string, it temporarily increases that, making it so the string may not be written to or modified by other threads.

That’d be totally relevant if Hellsinker were multithreaded. Which it’s not. Thanks Delphi!

So which strings are in the executable? The answer is a lot: All of the dialogue boxes in story mode, first-run segment 5 text, the Shrine of Farewell text, Tuning Dipsw descriptions, and the video filter list. The startup menu is a Windows resource. (And as of 1.0.0.9, ResHacker won’t work with it anymore, yay.)

Worse, the dialogue is stored completely out of order. (A box with lines 1,2,3,4 is stored as 2,4,3,1. yuck)

All of these had to be found, reorganized, translated, and put into our insertion script.

The hard work there was honestly finding it all. None of us even knew about the Shrine of Farewell text until the time came to analyze it.

Yeah, that’s text. Really. No, it’s not readable in Japanese either.

If you’re wondering, the particular text shown there is for Million Lives’ entrance:

No, no, no! You came all this way,
I'm the one who didn't respond!
Sorry, but I think I'm your opponent.

Given that we’re on a variable width font, it’s impossible to align to cell boundaries like it’s done in Japanese. Since you can’t read it there anyway, I don’t see this as being a big problem. Even the Japanese wiki extracted this text from the executable. So yeah, it’s only really in there for completeness’ sake.

There’s also a lot of unused text within the game. For example, it seems that Fossil Maiden originally had story dialogue for some segments, but it’s dummied out. Maybe it’s only there to further confuse people. Would not surprise me if that was its only purpose.

Font change

Left: Original. Middle: ‘System’ font in Wine. Right: Final font.

This is the set of hacks that underwent the most revisions. The original font used doesn’t work very well for English, so it had to be changed. After converting the text renderer over I set us on using the System raster font, which worked fine in my development environment.

Unfortunately, it looked different for everyone else who tried the test versions. The System raster font is different based on the target DPI for the screen used, with no manual way to set this, as stated by Microsoft, so that was out. None of the other Windows system fonts fit well enough for us, as It was important that we retain the angular, technical look of the original interface. Eventually, thanks to our trusty image editor Hal, we set on using a ttf font from dafont. I’m pretty sure we could do better, but it won out over the alternatives.

This meant we needed to load a custom ttf, of which there was no existing code in the game to do. Turns out the process for this is relatively simple:

- Load font from packfile (find and use existing packfile functions)
- Call GetModuleHandle on gdi32.dll
- Call GetProcAddress on gdi32.AddFontMemResourceEx
- Call AddFontMemResourceEx on font data

Relatively straightforward, but it had to be handwritten in assembly. This also meant figuring out how the pack functions worked, since they are only referenced directly by the game for the image and music code. Still, it only took a few hours.

For 1008f, the CreateFont calls had to be overridden and the loader placed on the first call. One CreateFont call had to be identified and skipped as it was used by the resource data for the startup menu, which we obviously did not want in that font.

Since the game would call CreateFont constantly throughout the game, we didn’t really need to do anything special here. The novel text uses a different font, which I’ll get into later, but that means there had to be some code to identify if it was from the novel part or anyplace else. Still, relatively simple.

For 1009, it only called CreateFont at program startup. This actually simplifies things but the loader hooks had to be completely rewritten to work on the main initialization function.

More complicated was the fact that this meant it used the same font reference for all text throughout the entire game. To recognize if which part of the program it was in, I had to change the handler to check the ExtTextOut call instead.

In both cases, I added a simple wrapper function around the novel code so that it would always know if it was in that part of the code or not.

Text centering

The first-run stage 5 text and the tuning dipsw screen both have centered text. Unfortunately, this being a Japanese game, it makes the assumption that it is monospaced text and just multiplies the string length by the number of characters. English does not look good in monospace, so this had to be fixed.

This ended with me hand-writing an assembly function to acquire the DC and font references to send to GetTextExtentPoint32. Extremely messy, but it was the only way.

Dialogue space expansion

Text space in the original and in English.

Although the play area is 360 pixels wide, the actual display part of the dialogue text is a miserable 256 pixels. This is fine for Japanese, but English takes up a lot more space, especially when translating a dense game like this, so we had overruns all over the place. So I set about expanding the text area, and while I succeeded it was a royal pain in the neck to do.

This ended up being way way way more of a hassle than I expected it to be, and it’s all because of how the text is stored. There’s a 1024×1024 texture slot it draws the new text regions to. The problem is where it puts them.As you can see, it lumps all of the dialogue lines into 256xwhatever strips in the texture, or all four on the same line. Obviously, it can’t be expanded sideways, so I had to reorganize the entire storage to 512xwhatever. This means that all the calls that were simple multiplications before needed to be changed to accessing tables to index where everything is.

In addition to this, the texture clearing functions had to be changed, so we didn’t leave any text behind on screen. This was handled differently in 1008f and 1009! Originally, the dialogue function had a clear function over the entire region used by the text, so I just expanded it. In 9 that’s been removed and tied to the text rendering call itself, which clears the defined zone. That took some effort to find.

There’s two separate dialogue rendering functions, too. One for the main text, and one for the fadeout. Both had to be modified, and each one had three functions that needed their parameters rewritten.

That was about a 10 hour hack all told.

Pain.

Image file resources

Fortunately for us, the images were very easy to work with and are just stored in a rudimentary packfile format that consists of a simple index and then unencrypted BMP files. Images that were RGBA were split into two images, one part for the RGB and one for the Alpha.

Interestingly, we found that the game does not like BMP files created by Photoshop. The files created by it add a few bytes of padding onto the end in excess of the size of the pixel area. While the game loads the entire section, padding and all, itonly allocates enough room for the pixel data. Memory overrun -> Crash. Whoops. I eventually hacked up my reinsertion tool to load PNG files and convert to the appropriate format.

Whenever any of the packfiles are accessed, instead of caching the index, it reloads the entire index from the file every time before scanning it. This is horrendously inefficient, but works out well for us because we can just point at a different packfile every time we needed to change something, allowing us to make bmp/tests_e.pak to contain everything.

By the way, the reason for the slow loading the game has is due to textures being converted to the current D3D PixelFormat type in quite possibly the slowest manner possible: one color channel at a time.

The images that we edited contain the segment 7 Rex dialogue, the segment 8 boss pattern text, the endings, and the main menu descriptions.

Image display realignment

Left: Jp/Vertical. Right: En/Horizontal.

Both the Rex text and the endings had to have the text converted from vertical to horizontal. This actually was pretty straightforward, simply change some numbers and convert subtractions to additions and I’m done.

For the endings, that was easy. For Rex, the problem was just finding the code in the first place. See, Hellsinker uses texture slots to store its data, and they are reused between every scene in the game. The slot that this text went into just so happened to be used by basically everything else.

This is only 5% of the references. Highlighted is the correct one.

As a bonus, the rendering code is nowhere even close to the loading functions, so I couldn’t cross-reference it that way.

I narrowed it down by basically taking a guess, given the structure of the data on the image, that the coordinate calculation code would have a specific SHL instruction in it. This brought the list down to a manageable size and I was able to find it within a couple hours. After that, changing it was a simple affair.

The novel text

The novel text is all contained in the text/ directory as .CTX files. They have some minor XOR encryption in place, but that’s easily foiled enough. Obviously replacing these files directly isn’t a big problem.

Of course, it couldn’t be that easy.

The text parser has a number of command codes, like <wait> to wait for you to press a button and <end> to finish that part up. But it’s not very bright and reads any word at the end of a line as a control code, so if there’s a line of dialogue that finishes with “end.” it ends the script.

Rather than try to figure out why it was grabbing stuff it shouldn’t, we just changed the keywords to unambiguous things: %wait and %end, in our case.

There was also a crash bug in some parts if you were advancing the text too quickly. This was caused by there only being a sanity check for the data on the main text advance, while the fast advance would happily explode itself on short strings. I fixed this by adding that check back in.

We use a different font for the novel text as the techy dialogue font just didn’t work out too well. I picked using the system font Lucida Sans Unicode and, well, nobody really had any complaints with it other than “too wide”, so it stuck. Even so, just in case we needed it I hacked the game to optionally load two ttf files, but that ended up unused.

The text redraw bug

Text redraw bug. Left: 1008f. – Middle: 1009. – Right: Correct.

The original Hellsinker game has had a known redraw bug with some video hardware. For whatever reason, the bug remains in the current version, but it’s not really acceptable for us to release a translation and then tell people to go to a wiki or something in order to actually read it. So I set out to figure out what the problem was.

For whatever reason, it turned out that the problem was that it cleared the video buffer with D3D calls and then rendered the text on top of it with WinGDI. Some video drivers like this, some don’t. When they don’t, no clearing occured.

For 1009 he set the text rendering to have an opaque background, so it would at least draw over the text that existed before, but it still wouldn’t clear anything that wasn’t drawn over.

Fixing this turned out to be pretty simple, just add a FillRect call immediately after the d3d clear. All better! In fact, this fixed another bug that I honestly did not expect to get fixed.

Left: Aliased. – Right: Antialiased.

I do all of my development on Linux and Wine, with no native Windows testing. So I was rather disheartened to hear that the text was not properly antialiased on Windows and generally just looked all wrong and ugly. Still, I didn’t have a clue why and there wasn’t much else I could do about it, so we kept it as is.

For whatever reason, fixing the background clearing bug also made the text magically start being antialiased correctly. While I’m not complaining about this turn of events, I really have absolutely no idea why. Looking at msdn gave me no hints. So I’ll just shake a fist at Windows and move on with my life.

Interestingly, 1009 also added a new text bug. The first run stage 5 dialogue only clears the portion drawn over to the right of the text, so the left side remains uncleared and gets drawn over. It also doesn’t hit the whole vertical section. I also fixed this bug.

What the heck are the spt files

Eventually, I came across a typo in one of the boss names, in particular “LOSTONE” from the Shrine of Farewell, that I felt should be fixed to “LOST ONE”. But this wasn’t contained in the executable, it wasn’t in any packfile, and it wasn’t in the novel text. So where the heck was it?

By the power of elimination, it had to be one of the spt files. I cracked open the fonts.spt file and started deciphering the format.

Turns out it consists of tile indices to bmp/fonts.bmp, which contains the font used to render most of the on-screen non-dialogue text. But the odd part is the order in which those glyphs are stored:

00000: G       ?       ?       L       
00080: R       ?       ?       O       
00100: E       ?       ?       S       
00180: A                       T       
00200: T       M       N       O       
00280:         I       I       N       
00300: I       L       N       E       
00380: M       L       E               
00400: P       I               T       
00480: U       O       L       E       
00500: R       N       I       M       
00580: E               V       P
...etc

Yeah, they’re stored vertically for no apparent reason. (the ? are just glyphs for [H/D], which are only three characters.) All the boss names and patterns are in there.

Amusingly, 1009 changed ‘SAINT MOUVE’ to ‘SAINT MAUVE’ in this file. It was the only change to fonts.spt. Despite that, the in-game dialogue still uses ‘mouve’, so I have no idea. We synchronized both to ‘mauve’ for the translation.

I think I can deal with being sainted.

The game does have a built-in converter from text to tile indices for that font image, so I really have absolutely no idea why this file exists. To perplex me, I guess.

Testing

Even with all this, we still need to actually test stuff. As there’s a lot of content in the game that is not quickly accessible, this can be a problem. Plus nobody’s actually going to try to clear the game every time they want to check something specific.

First thing that I did was add an invincibility hack. It’s not perfect, since you still get the penalties to Stella and so forth as if you were hit, but no actual death takes place. The actual changes to do this are two very simple executable patches:

3529db: change value 01 to 00
352b94: change value 01 to 00

Easy!

The endings were another part that needed to be warped to. There are seven in total and all of them need a run of the game up to at least segment 7, and testing all of them manually was just not going to happen.

Turns out that the ending screen is considered a main menu state! Possibly a leftover debug option. Either way, that makes it possible to just redirect any menu selection directly to the ending. Some additional hacking beyond that had to be done in order to determine which ending is acquired. There are four possible ending paths, and one of these changes dependent on the player character used:

Segment 8 - ed_sol_* - Satisfaction Lv.2 clear (character dependent)
Segment 8 - ed_conti - Satisfaction Lv.1 clear
Segment 7 - ed_luna - Spirit Overload ending (game over to boss's death explosion)
Segment 7 - ed_cruel - Spirit Overload ending (alternate)

I’m still not 100% sure of the conditions for the two segment 7 endings, personally. I just know they exist and need to be tested.

And finally the first run segment 5 text. This didn’t seem to be directly linked from any easily accessible part of the program, so I had to do a little more research to jump to it. That led up to me finding the relevant function to switch to the state that runs that code, and overriding some stuff so it would be called directly from the main menu code. It wasn’t perfect, but it worked well enough to get the bugs ironed out.

My final words:

Holy crap I am never doing a translation hack of this extent ever again.

That being said, I don’t regret working on this at all, even with the obvious cost to my sanity. So for now I’m going to take a break from this fan translating thing.

Either way, I hope you enjoyed the translation, either directly or indirectly.

Keep your dignity.

Leave a comment ?

10 Comments.

  1. I have no words.

    You crazy.

    Awesome, but crazy.

  2. On the topic of hex patches… The game’s numpad up-down controls are inverted for some strange gamepad-hugging-japanese reason. Could you by any chance post instructions how to fix that? (if not I’ll take a shot at it sometime).

    • No idea where that’d be, sorry.

      • 0038075C: 50 -> 48
        00380773: 48 -> 50

        So it turns out the game uses IDirectInputDevice8A::GetDeviceState() to poll the keyboard state, and uses a keymap to assign DIK_* values to buttons (32 physical + 8 fake for movement arrows). The map is initialized to some interesting values, and then significantly altered after clicking through the config screen. That’s where the mistake occurs. The above hex patch undoes that mistake (initially I just hooked that directinput call and did a double-swap).
        PS: In the process I fully identified the HsInput class that interfaces with DirectInput; if anyone wants the IDA .idb file, just poke me.

  3. Regarding your issues with the ending dialogues and main menu states: once you beat all of the game’s crazy additional stages, an additional option under “away” appears, called “inspect strategy recorder”. This is a viewer of all of the black screen monologues (with some additional graveyard stuff that appears once you read through them).

  4. I’m thinking of finding a way to alter the background + music played in the main menu (I just cleared everything, and the monochrome background + random bagpipe noise is de-motivating). You mentioned fiddling with menu states, did you by any chance find a toggle for that?

    • Uh, after you unlock everything you can change which menu background you use in Tuning DIPSW. Should be just above the return to main menu selection.

      • *dies* you’re right… well, I guess I’m done then :)
        about the only thing left on my wishlist is an unpacker for their .pak format (although I see that a generic file format ripper could sort-of do that already).

Leave a Comment